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Plain language summary 


Interventions can reduce school exclusion but the effect is temporary 


Some interventions — enhancement of academic skills, counselling, mentoring/monitoring, 
and skills training for teachers — appear to have significant effects on exclusion. 


The review in brief 


Interventions to reduce school exclusion are intended to mitigate the adverse effects of this 
school sanction. Some approaches, namely those involving enhancement of academic skills, 
counselling, mentoring/monitoring and those targeting skills training for teachers, have a 
temporary effect in reducing exclusion. More evaluations are needed to identify the most 
effective types of intervention; and whether similar effects are also found in different 
countries. 


Whatis the aim of this review? 


This Campbell systematic review examines the impact of interventions to reduce 
exclusion from school. School exclusion, also known as suspension in some 
countries, is a disciplinary sanction imposed by a responsible school authority, in 
reaction to students’ misbehaviour. Exclusion entails the removal of pupils from 
regular teaching for a period during which they are not allowed to be present in 
the classroom (in-school) or on school premises (out-of-school). In some 
extreme cases the student is not allowed to come back to the same school 
(expulsion). The review summarises findings from 37 reports covering nine 
different types of intervention. Most studies were from the USA, and the 
remainder from the UK. 
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What is this review about? 


School exclusion is associated with undesirable effects on developmental outcomes. It 
increases the likelihood of poor academic performance, antisocial behavior, and poor 
employment prospects. This school sanction disproportionally affects males, ethnic 
minorities, those who come from disadvantaged economic backgrounds, and those with 
special educational needs. 


This review assesses the effectiveness of programmes to reduce the prevalence of exclusion. 


What are the main findings of this review? 


What studies are included? 


Included studies evaluated school-based interventions or school-supported interventions to 
reduce the rates of exclusion. Interventions were implemented in mainstream schools and 
targeted school-aged children from four to 18, irrespective of nationality or social 
background. Only randomised controlled trials are included. 


The evidence base covers 37 studies. Thirty-three studies were from the USA, three from the 
UK, and for one study the country was not clear. 


School-based interventions cause a small and significant drop in exclusion rates during the 
first six months after intervention (on average), but this effect is not sustained. Interventions 
seemed to be more effective at reducing some types of exclusion such as expulsion and in- 
school exclusion. 


Four intervention types - enhancement of academic skills, counselling, mentoring/ 
monitoring, and skills training for teachers — had significant desirable effects on exclusion. 
However, the number of studies in each case is low, so this result needs to be treated with 
caution. 


There is no impact of the interventions on antisocial behaviour. 
Variations in effect sizes are not explained by participants’ characteristics, the theoretical 
basis of the interventions, or the quality of the intervention. Independent evaluator teams 


reported lower effect sizes than research teams who were also involved in the design and/or 
delivery of the intervention. 
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What do the findings of this review mean? 


School-based interventions are effective at reducing school exclusion immediately after, and 
for a few months after, the intervention (6 months on average). Four interventions presented 
promising and significant results in reducing exclusion, that is, enhancement of academic 
skills, counselling, mentoring/monitoring, skills training for teachers. However, since the 
number of studies for each sub-type of intervention was low, we suggest these results should 
be treated with caution. 


Most of the studies come from the USA. Evaluations are needed from other countries in 
which exclusion is common. Further research should take advantage of the possibility of 
conducting cluster-randomised controlled trials, whilst ensuring that the sample size is 
sufficiently large. 


How up-to-date is this review? 


The review authors searched for studies published up to December 2015. This Campbell 
systematic review was published in January 2018. 
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Executive Summary/Abstract 


BACKGROUND 


Schools are important institutions of formal social control (Maimon, Antonaccio, & French, 
2012). They are, apart from families, the primary social system in which individuals are 
socialised to follow specific codes of conduct. Violating these codes of conduct may result in 
some form of punishment. School punishment is normally accepted by families and students 
as a consequence of transgression, and in that sense school is often the place where children 
are first introduced to discipline, justice, or injustice (Whitford & Levine-Donnerstein, 2014). 


A wide range of punishments may be used in schools, from verbal reprimands to more 
serious actions such as detention, fixed term exclusion or even permanent exclusion from the 
mainstream education system. It must be said that in some way, these school sanctions 
resemble the penal system and its array of alternatives to punish those that break the law. 


School exclusion, also known as suspension in some countries, is defined as a disciplinary 
sanction imposed by a responsible school authority, in reaction to students’ misbehaviour. 
Exclusion entails the removal of pupils from regular teaching for a period during which they 
are not allowed to be present in the classroom or, in more serious cases, on school premises. 
Based on the previous definition, this review uses school exclusion and school 
suspension as synonyms, unless the contrary is explicitly stated. 


Most of the available research has found that exclusion correlates with subsequent negative 
sequels on developmental outcomes. Exclusion or suspension of students is associated with 
failure within the academic curriculum, aggravated antisocial behaviour, and an increased 
likelihood of involvement with punitive social control institutions (i.e., the Juvenile Justice 
System). In the long-term, opportunities for training and employment seem to be 
considerably reduced for those who have repeatedly been excluded. In addition to these 
negative correlated outcomes, previous evidence suggest that the exclusion of students 
involves a high economic cost for taxpayers and society. 
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Research from the last 20 years has concluded quite consistently that this disciplinary 
measure disproportionally targets males, ethnic minorities, those who come from 
disadvantaged economic backgrounds, and those presenting special educational needs. In 
other words, suspension affects the most vulnerable children in schools. 


Different programmes have attempted to reduce the prevalence of exclusion. Although some 
of them have shown promising results, so far, no comprehensive systematic review has 
examined these programmes’ overall effectiveness. 


OBJECTIVES 


The main goal of the present research is to systematically examine the available evidence for 
the effectiveness of different types of school-based interventions aimed at reducing 
disciplinary school exclusion. Secondary goals include comparing different approaches and 
identifying those that could potentially demonstrate larger and more significant effects. 


The research questions underlying this project are as follows: 


e Doschool-based programmes reduce the use of exclusionary sanctions in schools? 

e Are some school-based approaches more effective than others in reducing exclusionary 
sanctions? 

e Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school- 
based programmes on exclusionary sanctions in schools? 

e Docharacteristics of the interventions, implementation, and methodology affect the 
impact of school-based programmes on exclusionary sanctions in schools? 


SEARCH METHODS 


The authors conducted a comprehensive search to locate relevant studies reporting on the 
impact of school-based interventions on exclusion from 1980 onwards. Twenty-seven 
different databases were consulted, including databases that contained both published and 
unpublished literature. In addition, we contacted researchers in the field of school-exclusion 
for further recommendations of relevant studies; we also assessed citation lists from previous 
systematic and narrative reviews and research reports. Searches were conducted from 
September 1 to December 1, 2015. 


SELECTION CRITERIA 


The inclusion and exclusion criteria for manuscripts were defined before we started our 
searches. To be eligible, studies needed to have: evaluated school-based interventions or 
school-supported interventions intended to reduce the rates of suspension; seen the 
interventions as an alternative to exclusion; targeted school-aged children from four to 18 in 
mainstream schools irrespective of nationality or social background; and reported results of 
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interventions delivered from 1980 onwards. In terms of methodological design, we included 
randomised controlled trials only, with at least one experimental group and one control or 
placebo group. 


DATA COLLECTION AND ANALYSIS 


Initial searches produced a total of 42,749 references from 27 different electronic databases. 
After screening the title, abstract and key words, we kept 1,474 relevant hits. 22 additional 
manuscripts were identified through other sources (e.g., assessment of citation lists, 
contribution of authors). After removing duplicates, we ended up with a total of 517 
manuscripts. Two independent coders evaluated each report, to determine inclusion or 
exclusion. 


The second round of evaluation excluded 472 papers, with eight papers awaiting 
classification, and 37 studies kept for inclusion in meta-analysis. Two independent evaluators 
assessed all the included manuscripts for risk of quality bias by using EPOC tool. 


Due to the broad scope of our targeted programmes, meta-analysis was conducted under a 
random-effect model. We report the impact of the intervention using standardised 
differences of means, 95% confidence intervals along with the respective forest plots. Sub- 
group analysis and meta-regression were used for examining the impact of the programme. 
Funnel plots and Duval and Tweedie's trim-and-fill analysis were used to explore the effect of 
publication bias. 


RESULTS 


Based on our findings, interventions settled in school can produce a small and significant 
drop in exclusion rates (SMD=.30; 95% CI .20 to .41; p<.001). This means that those 
participating in interventions are less likely to be suspended than those allocated to 
control/placebo groups. These results are based on measures of impact collected immediately 
during the first six months after treatment (on average). When the impact was tested in the 
long-term (i.e., 12 or more months after treatment), the effects of the interventions were not 
sustained. In fact, there was a substantive reduction in the impact of school-based 
programmes (SMD=.15; 95%CI -.06 to .35), and it was no longer statistically significant. 


We ran analysis testing the impact of school-based interventions on different types of 
exclusion. Evidence suggests that interventions are more effective at reducing expulsion and 
in-school exclusion than out-of-school exclusion. In fact, the impact of intervention in out-of- 
school exclusion was close to zero and not statistically significant. 


Nine different types of school-based interventions were identified across the 37 studies 
included in the review. Four of them presented favourable and significant results in reducing 
exclusion (i.e., enhancement of academic skills, counselling, mentoring/monitoring, skills 
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training for teachers). Since the number of studies for each sub-type of intervention was low, 
we suggest that results should be treated with caution. 


A priori defined moderators (i.e., participants’ characteristics, the theoretical basis of the 
interventions, and quality of the intervention) showed not to be effective at explaining the 
heterogeneity present in our results. Among three post-hoc moderators, the role of the 
evaluator was found to be significant: independent evaluator teams reported lower effect 
sizes than research teams who were also involved in the design and/or delivery of the 
intervention. 


Two researchers independently evaluated the quality of the evidence involved in this review 
by using the EPOC tool. Most of the studies did not present enough information for the 
judgement of quality bias. 


AUTHORS’ CONCLUSIONS 


The evidence suggests that school-based interventions are effective at reducing school 
exclusion immediately after, and for a few months after, the intervention. Some specific types 
of interventions show more promising and stable results than others, namely those involving 
mentoring/monitoring and those targeting skills training for teachers. However, based on the 
number of studies involved in our calculations, we suggest that results must be cautiously 
interpreted. Implications for policy and practice arising from our results are discussed. 


12 The Campbell Collaboration | www.campbellcollaboration.org 


1. Background 


11 THE PROBLEM, CONDITION OR ISSUE 


1.1.1. School discipline 


Discipline problems are frequent in schools and they may have a harmful effect on pupils’ 
learning outcomes. A lack of discipline and the subsequent potential increase in school 
disorder (e.g. bullying, substance misuse) can seriously threaten the quality of instruction 
that teachers provide, hamper pupils acquisition of academic skills and subsequently reduce 
their attachment to the education system (Gottfredson, Cook, & Na, 2012). 


As such, discipline represents a serious concern for parents and teachers, demanding 
significant efforts and resources from schools (Kaplan, Gheen, & Midgley, 2002). The PISA 
2009 report (OECD, 2010) stated that schools registering higher levels of disciplinary 
problems result in teachers spending less time on learning in order to deal with such issues. 
In its 2012 version, the PISA report asked students about school discipline. Results found 
that “28% of students reported that teachers had to wait a long time to quiet down every 
class, or almost all classes” (OECD, 2013). Being more precise, the Teaching and Learning 
International Survey (TALIS) revealed that teachers spend at least 20% of lesson time 
dealing with disruption and administrative tasks. In the United Kingdom, the Behaviour 
Survey 2010 states that 80% of school teachers felt their ability to teach effectively was 
impaired by students’ poor behaviour (Massey, 2011). On a global level, evidence suggests 
that 13% of teachers’ time is, on average, spent maintaining order (OECD, 2009). 


Schools use different procedures to manage discipline, including a range of punitive 
responses (e.g., loss of privileges, additional homework or detention). Among these, 
exclusion is normally seen as one of the most serious punishments. Although types and 
lengths vary from country to country, school exclusion (also known as school suspension in 
the United States)! can be broadly defined as a disciplinary sanction imposed in reaction to 
students’ behaviour (i.e. violations of school policies) by the responsible authority. In 
concrete terms, exclusion entails a removal from regular teaching for a period of time during 


1JTn this document school exclusion and school suspension are used synonymously. 
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which students are also not allowed to be present on school premises. Specifically, fixed-term 
exclusions consist of a limited number of hours or days (Cornell, Gregory, & Fan, 2011), 
whereas permanent exclusion (i.e., expulsion) involves the pupil being transferred to a 
different school, or educated outside of the regular education system (Spink, 2011; Webb & 
Vulliamy, 2004). 


Even if school policies suggest that exclusion should be used as a last resort, reserved for only 
the most serious and persistent offences (Gregory & Weinstein, 2008a; Skiba & Peterson, 
1999; Skiba, Trachok, Chung, Baker, & Hughes, 2012), research evidence suggests that minor 
offences can also provoke this type of punishment (Munn, Cullen, Johnstone, & Lloyd, 2001; 
Skiba, 2014). Fenning et al., (2012) provide a case in point: their research concluded that 
suspension and expulsion were the most common types of punishment for minor problems 
such as tardiness and school truancy. These findings were also confirmed by Liu (2013) who 
found that 48% of suspensions lasting a maximum of five days targeted minor disorder or 
disruptive behaviours. 


In terms of prevalence, data provided by the UK Department for Education (academic year 
2011/12) shows that in England fixed-term exclusion affects 3.5% of the school population 
whereas permanent exclusion applies to only 0.06%. The national figures suggest that 
students in secondary-level education (6.8% of the school population) as well as those in 
special education (14.7%) are the most likely to experience fixed-term exclusion (DfE, 2013). 
In the United States, data provided by the Department of Education (academic year 2011/12) 
concluded that 7.4% (3.5 million) of students were suspended in school, 7% (3.45 million) 
were suspended out-of-school, and less than one per cent were subject to expulsion (around 
130,000 students). Black students and those presenting disabilities are, respectively, three 
and two times more likely to be excluded compared to White and non-disabled pupils (U.S. 
Department of Education, 2012). 


International comparisons of exclusion prevalence rates are not available in the literature 
examined. Indeed, differences in use, extent and recording (i.e., unreported exclusions) make 
an international estimation challenging. In Table 1, the reader will find information 
regarding the use of exclusion in a sample of high- and middle-income countries. The 
information is limited to a convenience sample involving twelve different cases to allow an 
overview of i) the types of exclusion used in these countries, ii) the length of the sanction, iii) 
the authority responsible for determining this sanction, iv) the behaviours for which school 
exclusion is permissible, and, in cases where information was available, the table also 
includes v) the local prevalence of exclusion. This does not claim to be a representative 
sample of all countries, but as an initial approach will help provide a more complex picture of 
the phenomenon. In addition, this comparison was intended to help with searches for studies 
that could be potentially included in the systematic review. For instance, by comparing 
exclusion in different countries, it was found that the same school sanction had different 
names in different countries (e.g., “stand-down” in New Zealand, “exclusion” in the UK and 
“suspension” in the US). 
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Table 1: Comparative description of school exclusion in a sample of high- and middle-income countries 


Country Name given Type of exclusions Length (for fixed Who makes the decision? — Legal reasons for exclusion Prevalence? 
exclusions) 
Australia? Suspension Short suspensions 4 school days. School Principal Continued disobedience and Unknown 
(New South Wales) aggressive behaviour 
Long suspensions Up to 20 school days. School Principal Physical violence, use or 


possession of prohibited 
weapons, firearms or knife, 
possession, use or supply of a 
suspected illegal substance, 
serious criminal behaviour, use a 
weapon, persistent or serious 
misbehaviour 


Expulsion Permanent Permanent School Principal In serious circumstances of 
misbehaviour, the principal may 
expel a student of any age from 
their school. The principal may 
also expel a student who is over 
17 years of age for unsatisfactory 
participation in learning. 


2 Prevalence was calculated by dividing the number of excluded students per year (numerator) by the total number of students per year (denominator). 
3 Information retrieved from “Suspension and Expulsion of School Students” New South Wales Government. Updated in October 2014 
https://www.det.nsw.edu.au/policies/student_serv/discipline/stu_discip_gov/suspol_o7.pdf 
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Country Name given Type of exclusions Length (for fixed 
exclusions) 
Canada‘ (Ontario) | Suspension Short-term 1 to 20 school days. More 
than five school days are 
Long-term considered long-term. 
Expulsion From school Indefinite. 


From all schools (in this 
case, the students must 
be offered alternative 
education) 


4 http://www.edu.gov.on.ca/eng/safeschools/NeedtoKnowSExp.pdf 
5 http://www.edu.gov.on.ca/eng/safeschools/statistics.html 
6 http://www.edu.gov.on.ca/eng/safeschools/statistics.html 


16 The Campbell Collaboration | www.campbellcollaboration.org 


Who makes the decision? 


School Principal. 

Parents must be informed 
within 24 hours. All 
suspensions can be appealed 
to the school board. 


School Principal should 
recommend expulsion to the 
school board. Parents must be 
informed within 24 hours. All 
expulsions can be appealed at 
a tribunal. 


Legal reasons for exclusion 


Threat to inflict serious bodily 
harm on another person, 
possessing alcohol or illegal 
drugs, being under the influence 
of alcohol, swearing at a teacher 
or at another person in a position 
of authority, committing an act of 
vandalism that causes extensive 
damage to school property, or 
bullying. 


Possessing or using a weapon, 
physical assault on another 
person that causes bodily harm 
requiring treatment by a medical 
practitioner, sexual assault, 
trafficking in weapons, trafficking 
in illegal drugs, robbery, drinking 
alcohol. 


Prevalence2 


2.16% of enrolled 
students 

(N= 2,014,407). 
Academic Year 2013- 
20145 


0.02% of enrolled 
students 

(N= 2,014,407) 
Academic Year 2013- 
20146 


Country Name given Type of exclusions Length (for fixed Who makes the decision? — Legal reasons for exclusion Prevalence? 
exclusions) 
Chile’ Suspension Fixed. Implemented The law does not limit the Disciplinary Board Defined for each school, but it Unknown 
inside the school duration of fixed must be used in exceptional 
premises suspensions. Each school cases 
community issues their 
own disciplinary code and 
defines disciplinary 
sanctions and their 
Expulsion duration. School Principal Conduct that directly threatens the Unknown 
physical or psychological integrity 
of any member of the school 
community® 
Colombia® Suspension Fixed Each school community Discretionary Violation to the code of conduct += Unknown 
issues their disciplinary 
codes and defines 
disciplinary sanctions and 
their duration. Normally 
fixed exclusion lasts 3 
Definitive days. Unknown 
Costa Rica‘? Suspension Fixed Up to 8 school days. School Principal Not clearly stated Unknown 
Permanent School Board Permanent disruptive/defiant 


behaviour, non-compliance with 
previous sanctions, violence and 
aggressions towards a member of 
the school community, lack of 
moral integrity. 


7 http://www.supereduc.cl/. Additionally, the information can be found in Torche & Mizala (2012) 
8 http: //denuncias.supereduc.cl/cuestionario1/denuncias_tematicos.html 
9 In Colombia, each school must define school exclusion length. This is established in the Ley General de Educacion N° 115, February 1994. 

http://www.mineducacion.gov.co/1621/articles-85906_archivo_pdf.pdf. Additional information can be retrieved from http://www.mineducacion.gov.co/1621/article- 


86906.html 


10 http: //www.mep.go.cr/sites/default/files/Reglamento_General_Establecimientos_Oficiales_de_Educacion_Media.pdf 
p Pp. g p 
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Country Name given Type of exclusions Length (for fixed Who makes the decision? — Legal reasons for exclusion Prevalence? 


exclusions) 
England" Exclusion Fixed (in-school, out-of- 1-45 days per year. After5 Discretionary Repeated failure to follow Academic year 2014- 
school) days of fixed out-of-school School principal academic instruction, failure to 2015'2 
exclusion, the school must complete a behavioural sanction 3.8% of students (all 
provide alternative (e.g. with a detention, adecision — schools) 
education. to change the sanction to 7.51% of students 
exclusion would not automatically (secondary schools) 
be unlawful), repeated and 
persistent breaches of the 
schools’ behavioural policy. 0.07% of students (all 
Permanent schools) 
0.15% of students 
(secondary schools) 
France's Exclusion Temporary exclusion Maximum of 8 days. Consultation between the Serious cases of violence Unknown 
from the classroom various members of the (physical or psychological) against 
pedagogical and educational _—_ the school community 
team 
Temporary exclusion Maximum of 8 days. School principal or school 
from school board 
Definitive exclusion Permanent Disciplinary board. The 


student should be represented 
on the disciplinary board. 


11 In England, exclusions are regulated by the Education Act of 2002 

12 https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/539704/SFR_26_2016_text.pdf 

13 In France, school exclusions are regulated by the Code de l’education: http://www.education.gouv.fr/cid56670/sanctions-scolaires-reforme-des-procedures-disciplinaires- 
dans-les-etablissements-scolaires.html 
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Country Name given Type of exclusions Length (for fixed Who makes the decision? — Legal reasons for exclusion Prevalence? 
exclusions) 
Finland‘* Exclusion In-school exclusion and __In-school exclusion: Teacher and school principal = Threats or serious violence that © Unknown 
out-of-school exclusion —_ remainder of the day. using a formal procedure. In would endanger the safety of 
with the school obligated Out-of-school exclusion: no cases of out-of-school other members of the school 
to provide education at more than 3 months. Itisa exclusion, a personal plan of | community 
home. very infrequent measure. education must be provided 
and local social services 
Permanent exclusion should be informed. 
does not exist in the 
local law. 
Malta Suspension Fixed term suspension Suspension for the rest of | Must be applied by the Head _—‘ The law defines 3 levels of Unknown 
the day or fora few days. of School after the student's = misbehaviour. Suspension and 
The number of days is not parent or guardian hasbeen __ expulsion are restricted for level 3, 
stated in the law. informed. The National Board meaning serious offenses only. 
for School Behaviour should —_No further details. 
be consulted. 
Expulsion Expulsion Permanent 
Norway *® Exclusion Fixed exclusion, Primary education (level 1- The school principal in Exclusion is used as a last resort Unknown 


expulsion for the rest of 
the year and loss of 
rights to education. 


7): exclusion from specific 
lessons or for the rest of 
the day. 

Secondary education (level 
8-10): maximum of 3 days. 
Expulsion and loss of 
rights are defined in the 
Educational Law but its 
use is extremely rare. 


consultation with the pupil's 
teacher, unless the local 
authority defines a different 
procedure. 


and can be justified only for 
serious issues of violence. The 
law suggests the use of 
alternatives such as mediation 
before imposing an exclusion. 


144 Basic Education Act 628/1998 (Amendments up to 1136/2010). http://www.finlex.fi/en/laki/kaannokset/1998/en19980628.pdf 


15 LOV 1998-07-17 nr 61: Law on Primary and Secondary Education (The Education Act) 
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Country Name given 

New Zealand ‘® Stand-down 
Suspension 
Exclusion 
Expulsion 


Type of exclusions 


Stand-down 


Suspension 


Exclusion 


Expulsion 


Length (for fixed 
exclusions) 


The student is removed 
from school for 5 school 
days in a term or 10 school 
days ina year. 


The student is removed 
from school for no more 
than 7 days. 

Maximum of 10 days ina 
year. 


A student under the age of 
16 would be excluded from 
the school, with the 
requirement that the 
student enrolls elsewhere 


A student aged 16 or over 
would be expelled from the 
school, and the student 
may or may not enroll at 
another school. 


Who makes the decision? 


School Principal, through a 
formal procedure that includes 
informing the family, the 
Education Authority and the 
school board. 


School Board 


School Board 


School Board 


Legal reasons for exclusion 


Drugs (including substance 
abuse), continual disobedience 
and physical assault on other 
students were the most prevalent 
causes for stand-down, 
suspensions, exclusion and 
expulsion. 


Prevalence2 


1.5% of school 
population (2015) 


0.3% of school 
population (2015) 


0.1% of the total 
student population 
under 16 years old 
(2015) 


0.2% of the total 
student population 
over 16 years old 
(2015) 1? 


16 In New Zealand, the guidance for suspensions is based on the Education Act of 1989 and the Education Rules 1999 (Stand-down, Suspension, Exclusion, and Expulsion) 
17 All data referring to prevalence was extracted from a governmental report informing data from academic years 2015. 
http://www.educationcounts.govt.nz/___data/assets/pdf_file/o011/147764/SSEE-Indicator-Report-2015-Data.pdf 
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Country 


The US, 
Washington DC ‘8 


Name given Type of exclusions 


Suspension Suspension (short-term 
and long-term) is a 
restriction in attending 
school or school 


activities. 


Expulsion Expulsion makes this 


restriction indefinite. 


Emergency expulsion 


Length (for fixed 
exclusions) 


Short-term suspension: 
maximum of 10 
consecutive days. 


Long-term suspension: 


more than 10 consecutive 


days. 


Maximum: 1 calendar year 


Temporary. The student 


would go back once the 
danger ceases 


Who makes the decision? — Legal reasons for exclusion 


Certified teachers can decide __ Violation of school district rules 
a suspension but it must be 
communicated to the school 
principal. Short-term 
suspensions must be formally 
communicated to the 
student/parents. Long-term 
suspensions and expulsions 
require a formal process (i.e., 
written notice by the school 
district) and should be known 
by the School Principal. 


Violation of school district rules, 
serious violence, gang activity on 
school grounds. 


The student's presence poses an 
immediate and continuing danger 
to others. The student's presence 
poses a threat of substantial 
disruption in the classroom. 


Prevalence2 


3.89% of all 
Washington students 
have been suspended 
or expelled (2014-15) 
The rate of 
suspensions and 
expulsions across 
districts range 
between nearly 0% to 
over 10% of 
students "9. 


18 Tn the US, procedures and definitions of school suspension vary among states. Here, we use Washington State as an example. For more details, see 
www.k12.wa.us/Safetycenter/Discipline/pubdocs/Suspension-expulsion-rights.pdf 

19 Data extracted from Office of Super Intendent of Education (OSIP), State of Washington. 
http://www.k12.wa.us/DataAdmin/PerformanceIndicators/DataAnalytics.aspx#discipline 
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Country 


The US, 
Virginia2° 


Name given Type of exclusions Length (for fixed 
exclusions) 

Removal from In-school 

classes 

Suspension Suspension (short-term Short-term suspension: 10 
and long-term) is a consecutive or 10 
restriction in attending cumulative school days in 
school or school a school year 
activities. Long-term suspension: 


more than 10 school days 
but less than 365 calendar 
days. 

Expulsion Expulsion makes the A student is not permitted 

restriction last longer. to attend school within the 

school division and is 
ineligible for readmission 
for 365 calendar days after 
expulsion. 


20 See the specific section for Virginia, p. 10-16 in 
https://safesupportivelearning.ed.gov/sites/default/files/disciplinecompendium/School%2oDiscipline%20Laws%20and%20Regulations%20Compendium.pdf 
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Who makes the decision? 


Teacher 


Imposed by the school 
principal, any assistant 
principal or, in their absence, 
any teacher. The suspension 


should entail a formal process. 


The student must be heard. 


Imposed by a committee from 
the school board. Includes a 
formal process, written notice 
and appeal. 


Legal reasons for exclusion 


Disruptive behaviour 


Violation of school code of 
conduct 


Criminal activity, carrying a 
weapon, drug related offences, or 
when the pupil presence is a clear 
threat for the school community. 


Prevalence2 


Unknown 


Country Name given Type of exclusions 


Length (for fixed 
exclusions) 


Who makes the decision? — Legal reasons for exclusion 


Low-level offences are dealt Violation of school code of 


Prevalence2 


The US, Texas?! Suspension In-school suspension 


(e.g., seclusion units) 


Expulsion Out-of-school 
suspension 


In the case of serious 
offences, a student can 
be expelled from school. 


In-school suspension lasts 
between 1 class and 
several days. 


Out-of-school suspension 
should be no longer than 3 
days. 


At least 1 year 
Disciplinary Alternative 
Education Program 
(DAEP) for students 
removed for over 3 days 
(no maximum period 
provided). 


with on a discretionary basis conduct (unruly, disruptive, or 
(according to a defined code of abusive behaviours) 

conduct) by the designated 

administrator (usually the 

principal or vice principal). 

Higher-level offences require 

mandatory removal from the 

classroom. Rules for a due 

process are defined. 


Unknown Weapon carrying, serious 
violence or crimes. 


Unknown 


9.24% (2014-2015) 24 


4.33% (2014-2015) 


3.39% (2014-2015) 


21 See the specific section for Texas, p. 14-27 in https://safesupportivelearning.ed.gov/sites/default/files/discipline- 
compendium/School%2oDiscipline%20Laws%20and%20Regulations%20Compendium.pdf 
22 Data extracted from the Texas Education Agency based on categories which count students once. https://rptsvri.tea.texas.gov/cgi/sas/broker 
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The comparative data incorporated in the table above suggests heterogeneity in the 
application of exclusion. For instance, in the US, Norway and England, educational systems 
distinguish between fixed and permanent exclusion. However, in some educational systems, 
such as Finland’s, the law only permits fixed-term exclusion. Concerning length, England 
limits fixed-term exclusions to a maximum of 45 days per school year while New Zealand’s 
legislation allows exclusions for a maximum of 10 days per year. On the other hand, it is 
important to note that in some countries — such as France — specific laws define and regulate 
exclusion, whereas in others — like Chile and Colombia — the ability to set the length of the 
sanction is granted to each school. 


Although the data on prevalence is limited to a few countries, the percentage of in-school 
exclusion seems to be larger than out-of-school exclusion and expulsion. In New Zealand, the 
data suggests that the use of exclusion is marginal particularly when compared with some 
areas in the US and the UK. 


12 PREDICTORS AND OUTCOMES 


The research concerning predictors and outcomes of school exclusion has some limitations it is 
necessary to address before arriving at any final conclusions. Regarding predictors, only 
ethnicity seems to have a clear role in predicting exclusion. For other variables of interest such 
as Sex, age or socio-economic status most of the evidence is limited to bivariate associations. 


Regarding the outcomes, while there is a stark link between misbehaviour (e.g., school 
drop-out and delinquency) and school exclusion, there is no clear causal relationship. 
Notwithstanding decades of research on school exclusion and its impact on later behaviour, 
we are still at an initial stage for testing causal associations in these matters. The 
association between exclusion and these negative outcomes may simply reflect underlying 
behavioural tendencies that lead to conduct problems, exclusion and poor outcomes later in 
life — that is, the antisocial syndrome described by Farrington (1997). In fact, school 
exclusion and the behaviours outlined here as “negatives” could be explained by the 
personality traits of the syndrome. 


As stated by Sutherland and Eisner (2014) “it is currently unclear whether the disciplinary 
action itself has a causal effect over and beyond the social, familial and behavioural 
characteristics of the affected children. To date, studies have used analytical approaches that 
are unable to reliably establish a robust link between exclusion and outcomes such as criminal 
behavior.” Some longitudinal studies have attempted to deal with this matter by controlling for 
previous behavioural characteristics that could alter the impact of the sanction. When this is 
the case, the methodological details are explicitly presented in this review. 


Keeping these reservations in mind, the following section describes variables associated with 
the prediction of school exclusion, as well as some negative outcomes linked to exclusion. 
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1.2.1 Predictors of school exclusion 


From a normative point of view, school exclusion is a punitive response for misbehaviour. In 
that sense, behavioural problems seem to be the most obvious empirical predictor for 
exclusion. Reinke, Herman, Petras, & Ialongo, (2008) illustrate the role of problem 
behaviour in exclusion by conducting a latent class analysis. Participants in the subclass of 
boys exhibiting behavioural problems only (..e., isolating other academic/learning 
difficulties) were almost 4 times more likely to be suspended (OR = 3.42; 95%CT 1.36 to 8.58; 
p < .05) than their non-problematic peers. Similarly, Pas, Bradshaw, Hershfeldt, & Leaf, 
(2010) found that after controlling for student, teacher, classroom, and school level 
covariates, the strongest predictor for out-of-school suspension was disruptive behaviour 
(OR = 4.83 ; 95%CI 4.10 to 5.68; p < .05). 


Despite the role of behaviour in school exclusion, research suggests that it is not the sole or 
even the most prominent predictor. In fact, previous findings show a more complex scenario 
where exclusion is also strongly predicted by gender, ethnicity, age, economic background, 
and special educational needs (Costenbader & Markson, 1998a; Mcloughlin & Noltemeyer, 
2010; Monroe, 2005; Nickerson & Spears, 2007; Noltemeyer & Mcloughlin, 2010; Skiba et 
al., 2011; Yudof, 1975). In the following paragraphs, we offer an overview of the role of these 
variables in predicting school exclusion. 


Gender as a predictor of exclusion 


Data provided by the Department for Education in England (DfE) 2011/12 suggests that male 
pupils are around three times more likely to be punished by exclusion than female pupils 
(DfE, 2013). The same trend can be observed in the study published by Liu (2013) based on 
longitudinal data from 13,875 American students. The study reports the predominance of 
males being excluded, but recognises that the proportion of females excluded tends to 
increase from elementary (23.7%), to secondary (32.7%), to high school (35.2%). More 
specifically, Bowman-Perrott et al. (2013, p. 91) concluded that, based on a sample of 2,597 
pupils, the predominance of males in exclusion rates (OR = 2.28) was even larger in the case 
of pupils with learning disabilities (OR = 4.31).23 


Ethnicity 


Research outcomes suggest a clear and consistent disproportionality in the prevalence of 
ethnic minorities as a target for disciplinary exclusion (Anyon et al., 2014; Gregory, Skiba, & 
Noguera, 2010). In the US, different sources of data show that school exclusion overly affects 
minorities such as Afro-Caribbean (Noltemeyer & Mcloughlin, 2010), Latino (Skiba et al., 
2011) and American Indian students (Gregory et al., 2010) in comparison with their White 
peers. In the UK, data from the (DfE, 2012) showed that: “The rate of exclusions was highest 
for Travellers of Irish Heritage, Black Caribbean and Gypsy/Roman ethnic groups. Black 
Caribbean pupils were nearly 4 times more likely to receive a permanent exclusion than the 


23 Odds ratios are presented without confidence intervals or p-values as in the source. 
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school population as a whole and were twice as likely to receive a fixed period exclusion.” 
Notably, recent multivariate analysis points out that racial disproportionality in exclusion 
still remains significant after controlling by behaviour, number and type of school offences, 
age, gender, teacher’s ethnicity, and socio-economic status (Fabelo et al., 2011; Noltemeyer & 
Mcloughlin, 2010; Rocque & Paternoster, 2011; Skiba, Michael, Nardo, & Peterson, 2002). 
Consider, for instance, a substantial longitudinal report produced by Fabelo et al., (2011) in 
Texas (N=928,940), intended to isolate the effect of race alone on disciplinary actions. The 
study used a multivariate analysis controlling for 83 different variables. The findings suggest 
that African-American students were 31% more likely to be removed from classrooms 
compared to White and Hispanic students. In the same vein, Skiba (2015) has argued that, in 
the United States at least, racial disproportionality in school discipline is ubiquitous. In his 
opinion, ethnic minorities are overrepresented in almost all types of school punishment. 
Even more worryingly, instances of exclusionary discipline among African Americans have 
continued to increase over the years. 


Possible reasons for this overrepresentation of Black students, even when controlling for 
demographic and risk factors, have been addresses by some scholars, who suggest that a 
racist bias could explain the phenomenon (Losen, 2011; Skiba et al., 2002; Skiba, 2015). In 
particular, Simson (2014) asserts that racial stereotyping (conscious or unconscious) as well 
as a cultural mismatch between teachers and students can explain at least some part of the 
existing racial disproportionality in school discipline. 


It is important to say that, as stated by Theriot, Craun, & Dupper (2010, p.14), “the over- 
representation of ethnic minority students, especially African American students, in school 
suspension and expulsion is one of the most consistent—and perhaps most controversial— 
findings in the extant literature on school discipline.” In general, studies using solid and 
strong multivariate models highlight the discrimination against racial minorities compared 
to White students. 


Age as a predictor of exclusion 


The likelihood of being punished by exclusion increases with age, being more frequent during 
adolescence. In England, 52% of permanent exclusions are imposed on pupils aged between 
13 and 14 (DfE, 2013). In the case of American students, the results follow a similar trend. In 
fact, data reported by Liu, (2013) pointed out that suspensions reach a peak in ninth grade 
(i.e., 14 to 15 years of age). Also based on a sample of American students, Raush & Skiba, 
(2004) concluded that the number of out-of-school suspensions was significantly higher in 
secondary schools compared to elementary schools. 


Socio-economic status (SES) 


Low SES has also been identified as a predictor of high rates of disciplinary exclusion. The 
UK Department of Education (DfE, 2012) compared the rates of exclusion by eligibility for 
free school meals (FSM). Those eligible for FSM were 4 times more likely to be punished by a 
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permanent exclusion and around 3 times more likely to get a fixed-period exclusion than 
children who were not eligible. In the US, Nichols, (2004) using a sample of 52 schools 
(37,000 students), found a similar pattern — but the correlation between FSM and exclusion 
was higher and more significant for pupils in middle school (r = .84; p <. 01) than for 
elementary (r = -.12) or high school pupils (r = .48). In Australia, Hemphill et al., (2010), 
using multilevel mixed-effects logistic regression (N = 8,028 students), concluded that pupils 
settled in low SES neighbourhoods were exposed to higher rates of exclusion (8.7%) when 
compared with pupils in high SES areas (2.9%). 


However, the evidence still seems to be inconclusive in this respect. Recently, Skiba et al., 
(2012), using a multilevel approach, tested data from 365 schools and a total number of 43,320 
students. They concluded that when comparing those students eligible for free or reduced-cost 
lunches with their non-eligible peers, the first were more likely to get out-of-school exclusions 
(OR= 1.27; p<.05). However, contrary to expectations, the eligibility for free or reduced meals 
resulted in a negative predictor of permanent exclusion (OR= 0.03; p<.05). 


Special educational needs (SEN) 


Although an increasing amount of research has focused on predictors of school exclusion, 
analysis of the role of SEN still seems to be limited. In 2007, Achilles, McLaughlin, and 
Croninger differentiated the role of three different SEN, namely emotional/behavioural 
disorders (EBD), attention-deficit/hyperactivity disorders (ADHD), and learning disabilities 
(LD). Higher rates of exclusion were more likely among those with EBD (OR = 1.49; p<. 001) 
compared with ADHD (OR = 2.58; p < .001) or LD (OR = 5.44; p < .001). Recently, Bowman- 
Perrott et al. (2013), using three waves from the Special Education Elementary Longitudinal 
Study (SEELS), confirmed that children with emotional or behavioural disorders (OR = 3.95; 
p <.05) and attention-deficit or hyperactivity disorders (OR = 4.96; p <.05) were more likely 
to get suspended or expelled from school than children with learning disabilities (OR = 2.54; 
p <.05). In a study involving 2,750 students and 39 American schools, Sullivan, Van Norman, 
& Klingbeil (2014) also observed differences between types of disabilities: those presenting 
an EBD were at a far greater risk of exclusion (OR = 6.78; SE=0.21) than those presenting 
other health impairments (i.e., a specific learning disability, intellectual disability, speech 
and language impairment). When controlling for race and gender, and parents’ education, 
this trend remained stable and significant. It is important to emphasize that the associations 
between this disability and exclusion mainly reflect differences in behaviour, respectively 
psychological or chronic behavioural problems. 


1.2.2 Negative outcomes linked to school exclusion 


Supporters of zero tolerance policies have pointed out that the use of exclusion can persuade 
students to account for their behaviour and lead to a decrease in rule-breaking (Bear, 2012). 
However, most of the research has consistently documented the negative impact of these 
types of sanctions (APA Zero Tolerance Task Force, 2008; Chin, Dowdy, Jimerson, & Rime, 
2012; Hemphill, Toumbourou, Herrenkohl, McMorris, & Catalano, 2006; Sharkey & 
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Fenning, 2012). In particular, previous research suggests that school exclusion is related to 
serious negative outcomes in at least three dimensions of young people’s development: 
behavioural, academic, and future social inclusion. 


Behaviour 


Some literature related to the relationship between exclusionary punishments and behaviour 
suggests that such harsh punishments could result in a spiral into more defiant behaviour by 
students. Raffaele-Mendez, (2003), for instance, found a moderate and significant 
correlation (r = .39) between out-of-school exclusion (grades 4 to 5) and subsequent 
exclusion (grade 6). Similarly, Theriot, Craun, & Dupper, (2010) found that pupils punished 
by in-school and out-of-school exclusion were slightly more likely to get the same 
punishment again (ORin-school = 1.253 D < .001; N = 9706 and ORouwt-of-school = 1.323 Pp < .001; N = 
9706). 


Using longitudinal data, Arcia, (2006:366) concluded that school dropout was another 
behavioural consequence of exclusion. In fact, “43% of students who were suspended 21 or 
more days dropped out 3 years after their ninth-grade enrolment.” Similarly, Cratty 
(2012:649) found a positive correlation between out-of-school suspensions and dropout 
rates. In particular, “those who had an early record of multiple exclusions registered 60% 
dropout during high school” when compared with non-excluded students. 


The use of exclusion, in turn, is linked with more serious behavioural outcomes such as 
antisocial conduct, delinquency and entry into the juvenile justice system. Longitudinal 
research carried out by Hemphill et al. (2006:736) argues that “school suspensions 
significantly increased antisocial behaviour 12 months later, after holding constant 
established risk and protective factors (OR = 1.5; 95%CI 1.1 to 2.1; p < .05; N = 3655)” In 
terms of the involvement of school excludees in the criminal justice system, Costenbader & 
Markson, (1998) found significant differences between excluded students and those never 
excluded. In their view, “while 6% of the students who had never been suspended reported 
having been arrested, on probation, or on parole, 32% of the externally suspended subsample 
and 14% of the internally suspended subsample responded positively to this question. Males 
reported significantly more involvement with the legal system than did females.” (p.67). 
Meanwhile, Challen & Walton, (2004), studying a population of males in the criminal justice 
system, concluded that more than 80% had been previously excluded from school?4. 


Academic achievements 


Evidence suggests that periods of exclusion may have detrimental effects on pupils’ learning 
outcomes. Exclusion is accompanied by missed academic activities, alienation, and 


24 The strong link between school exclusion and subsequent school drop-out/entry into the juvenile justice system 
has been termed the “school-to-prison pipeline” (Snyder & Sickmund, 2006; Fenning et al., 2012; Chin et al., 2012 
Christle et al., 2005; Nicholson- Crotty et al., 2009). It describes the escalating punitive consequences of harsh 
discipline in school and its exclusionary practices. 
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demotivation in relation to academic goals (Brown, 2007; Michail, 2011). In particular, 
Hemphill et al., (2006) found that excluded pupils were slightly more prone to fail in the 
academic curriculum when compared with non-excluded students (OR = 1.3, 95% CI 1.1 to 
1.5, p < .01). Along similar lines, Arcia, (2006) produced a longitudinal retrospective study 
regarding the associations between exclusions and achievements from fourth to seventh 
grade. After three years, non-excluded students displayed substantially higher reading 
achievement scores when compared with their non-excluded peers. In fact, seventh-grade 
students who were excluded for 21 days or more achieved scores similar to fourth-grade 
students that had not been excluded. Finally, Raffaele-Mendez, (2003) added that those 
excluded were also less likely to graduate from high school on schedule. 


Future social inclusion 


Some studies have pointed out that young people excluded from school can also register a 
high risk of becoming “Not in Education, Employment, or Training” (NEET) in the future. In 
2007, Brookes, Goodall, & Heady stated that students who had been excluded were 37% 
more likely to be unemployed during adulthood. Spielhofer et al. (2009) showed that among 
individuals with long-term status as NEET, the majority had previous experienced of 
exclusions and truancy. More precisely, Massey (2011) argued that approximately one out of 
two excluded children will be NEET within two years of their exclusion. 


Research has also illustrated the long-term implications of exclusion for society as a whole. 
In economic terms, the cost of excluding children from school places a demand on public 
resources. Although the literature on this matter is still limited, Brookes et al. (2007) 
produced a report regarding the costs of permanent exclusion in the United Kingdom. The 
analysis encompasses an estimation of costs for the individual as well as for the educational, 
health, social and criminal justice services. Overall the cost, in 2005 prices, of permanently 
excluding a student was estimated at £63,851 per year to society. 


While there is a stark link between the aforementioned negative outcomes and school 
exclusion, these should not be regarded as causal. Notwithstanding decades of research on 
school exclusion and its impact on later behaviour, we are still at an initial stage for testing 
causal associations in these matters. The association between exclusion and these negative 
outcomes may simply reflect underlying behavioural tendencies that lead to conduct 
problems, exclusion and poor outcomes later in life — that is, the antisocial syndrome 
depicted by Farrington (1997). In fact, school exclusion and the behaviours described here as 
“negative outcomes” could be explained by the same underlying factors or personality traits 
characterising the syndrome. 


Despite the lack of empirical support for a causal association, some criminological theories 
provide a plausible explanatory framework to understand the connection between 
punishment and the persistence of deviant behaviour. Labelling theory, for example, suggests 
that those punished (by exclusion) and labelled as “deviant” may start behaving in ways that 
conform to their newly formed self-image: by limiting their interactions with integrated 
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students, for example, and shunning conventional social systems such as school (Krohn, 
Lopes, & Ward, 2014, p. 179). Likewise, Sherman's defiance theory (1993) elucidates the 
circumstances in which a punishment can produce more antisocial behaviour, such as 
defiance, instead of compliance with rules. In his view, punishment can increase the 
prevalence, incidence or seriousness of future offending when offenders deny responsibility, 
and when they perceive sanctions as unfair, stigmatising and imposed by an illegitimate 
authority. 


Finally, in addition to all these findings and the rationale around the negative outcomes 
linked to school exclusion, it is important to mention that, so far, there is no evidence 
demonstrating that exclusion is effective for improving school discipline (Skiba, 2014). What 
is more, in the short term, exclusion seems to directly deny students’ right to access 
education as well as reducing adult supervision for those who are most at risk of further 
deviant behaviour, or most in need of teachers’ support. 
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13 THE INTERVENTION 


1.3.1 School-based programmes 


The prevalence of exclusion and its adverse correlated consequences have caught the 
attention of policy makers and programme developers. As a result, a range of interventions 
have been designed and implemented to improve school discipline. In the present review, we 
include different types of school-based intervention aimed at reducing school exclusion as a 
punishment for inappropriate behaviour. These interventions include those targeting 
individual risk factors or school-related factors, as well as those using a more comprehensive 
strategy that includes parents, teachers, school administrators, and the community. 


Interventions targeting individual risk factors include, for instance, cognitive-behavioural 
approaches such as anger management programmes or skills training for children (e.g., 
Humphrey & Brooks, 2006). Another type of intervention focusing on student behaviour — 
or, more precisely, students’ skills for conflict resolution — are restorative justice 
programmes (e.g., Schellenberg & Parks-Savage, 2007; Shapiro, Burgoon, Welker, & Clough, 
2002) In general, these interventions target motivated children and train them in practical 
skills to deal with anger, solve conflicts or become more assertive in social relationships. Such 
interventions are normally organised within a curriculum and implemented during school 
hours. The curriculum involves a package of group or one-to-one sessions using a wide range 
of techniques such as instruction, modelling, role-play, feedback, and reinforcement, among 
others (Gottfredson, Cook, & Na, 2012; Schindler & Yoshikawa, 2012). 


At the classroom level, interventions may target teachers’ abilities in classroom management 
(Pane, Rocco, Miller, & Salmon, 2013). The training for teachers encompasses instructional 
skills, such as guidelines for teaching rules and maintaining attendance, and non- 
instructional skills, such as group management techniques, reinforcing positive conduct, and 
techniques to explain expected behaviour. Both skill sets are aimed at improving the learning 
process, preventing misbehaviour and encouraging positive participation by pupils (Averdijk, 
Eisner, Luciano, Valdebenito, & Obsuth, 2014). 


Some schools offer mental health services independently or via community agencies. 
Experienced clinicians are located in schools in order to deliver individual, group, and/or 
family therapy. Clinicians may also be available for teacher consultation on matters related to 
students’ behavioural and emotional issues. All these interventions may target a reduction in 
out-of-school exclusion (Bruns, Moore, Stephan, Pruitt, & Weist, 2005). 


Alternatively, comprehensive prevention strategies target students, families, teachers and 
school managers as well as the community as a whole (Bradshaw, Waasdorp, & Leaf, 2012; 
Flay & Allred, 2003; Colin Pritchard & Williams, 2001; Snyder et al., 2010). A well-known 
comprehensive programme is the School-Wide Positive Behavioural Interventions and 
Supports (SWPBIS). The programme aims to provide support for positive conduct by 
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building proactive school-wide disciplinary procedures (i.e., improving school climate and 
reducing problem behaviours). SWPBIS incorporates a multi-level approach: from whole 
school prevention, to group-based intervention for problematic pupils, and personalised, 
tailored interventions for high-risk students. The basic elements of the programme are: i) 
building a school culture for both social and academic attainment, ii) early prevention of 
problem behaviours, iii) teaching social skills to all students, iv) using behaviour support 
practices, and v) actively using data for decision-making. Research reports promising results, 
although further and stronger evaluation designs need to be undertaken (Gottfredson et al., 
2012; Maag, 2012). 


Previous reviews 


In 2013/14 we conducted a systematic search of reviews and meta-analyses assessing the 
effectiveness of school-based programmes for promoting early prevention of risks (Averdijk 
et al., 2014). The results suggested there had been no previous meta-analysis aimed at 
assessing the effectiveness of different types of interventions for reducing disciplinary school 
exclusion. Probably the most similar study is one published by Burrell, Zirbel, & Allen, 
(2003) who conducted a meta-analysis on the effectiveness of mediation programmes in 
educational settings. Among many other outcomes, the analysis suggested that these 
interventions had a desirable effect (r = -.287, k = 17, N = 5,706, p < .05) on administrative 
suspensions, expulsions and disciplinary actions. However, in this meta-analysis suspension 
was reported along with other disciplinary actions, and the study did not compare mediation 
with any other intervention (as proposed in the present meta-analysis). The authors also call 
for a cautious interpretation given the high heterogeneity of primary results. A similar type of 
analysis was followed by Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, (2011) and 
Gottfredson, Wilson, & Najaka, (2002). In both studies, school exclusion was coded as an 
outcome, but the final meta-analysis did not report on the impact of the intervention 
specifically in relation to this targeted outcome. 


Likewise, Solomon, Klein, Hintze, Cressey, & Peller, (2012) conducted a meta-analysis 
exclusively testing the effectiveness of School-Wide Positive Behavioural Interventions and 
Supports (SWPBIS) programme. Despite a small number of included studies reporting data 
on exclusion, the review does not report effect sizes by measuring their increase/decrease. 
Rather, the review reports effect sizes on the reduction of office discipline referrals and 
problematic behaviour. 


In addition, two narrative reviews have recently been produced looking at intervention as a 
means of reducing disciplinary exclusion. Spink, (2011) explored qualitative, quantitative and 
mixed methods studies. Overall, 10 reports were found. The review concluded that multi- 
agency interventions were the most frequent and that they could have a positive effect on 
reducing exclusion of pupils who are at risk. As expected, the study did not report a meta- 
analysis of effect sizes. In 2012, Johnson produced another narrative review identifying 
programmes that may be an alternative for suspension in school systems. The search 
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strategies were not clear enough to allow replication and, again, the nature of the design does 
not allow for the calculation of effect sizes. 


1.4 WHY IT IS IMPORTANT TO DO THE REVIEW 


Despite a growing body of research on the negative side effects of exclusion, no previous 
meta-analysis based on a comprehensive systematic review has been conducted to synthesize 
evidence assessing the impact of school-based interventions in reducing disciplinary 
exclusion. The current review addresses this gap by meta-analysing results from existing 
published and unpublished studies, providing a statistical assessment of the overall effect of 
school-based interventions at reducing exclusion. 


This meta-analytic investigation has clear implications for policy making. The results 
provided by the present study would produce a much-needed evidence base for school 
managers, policymakers and researchers alike. These results can contribute to tackling the 
adverse developmental, social and economic effects of school exclusion mentioned in the 
previous pages, as well as potentially identifying alternative and less punitive approaches to 
school discipline. 
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2. Objectives 


The main goal of the present research is to systematically examine the available evidence for 
the effectiveness of different types of school-based interventions for reducing disciplinary 
school exclusion. Secondary goals include comparing different types of interventions (e.g., 
school-wide management, classroom management, restorative justice, cognitive-behavioural 
interventions) and identifying those that could potentially demonstrate larger and more 
significant effects. 


We also aim — potentially — to run analysis controlling for characteristics of participants 
(e.g., age, ethnicity, level of risk); interventions (e.g., theoretical bases, components); 
implementation (e.g., facilitators’ training, doses, quality), and methodology (e.g., research 
design). 


The research questions underlying this project are as follows: 


e Doschool-based programmes reduce the use of exclusionary sanctions in schools? 

e Are some school-based approaches more effective than others in reducing exclusionary 
sanctions? 

e Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school- 
based programmes on exclusionary sanctions in schools? 

e Docharacteristics of the interventions, implementation, and methodology affect the 
impact of school-based programmes on exclusionary sanctions in schools? 
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3. Methods 


3.1 TITLE REGISTRATION AND REVIEW PROTOCOL 


The title of the present review was registered in The Campbell Collaboration Library of 
Systematic Reviews on January 2015. The final version of the review protocol was approved 
in November 2015. The title registration and the respective protocol are available at: 
https://www.campbellcollaboration.org/library/reducing-school-exclusion-school-based- 
interventions.html 


3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW 


3.2.1. Research design 


Our original proposal was to include both randomised controlled trials and high-quality 
quasi-experimental studies (defined as studies using a comparison group, pre-post testing 
and a statistical matching approach). To be eligible for inclusion, we stated that manuscripts 
must clearly report the method used to ensure equivalence between treatment and control 
groups, taking into account major risk factors (e.g. behavioural measures) and demographic 
characteristics.25 


In this review, we only present results from randomised controlled trials (RCTs). There were 
three reasons for our decision: 


1. First, even though a number of quasi-experimental studies initially fulfilling our 
inclusion criteria were found by our searches, many of them fail to report baseline 
measures (e.g., Guardino, 2013; Hawkins, Catalano, Kosterman, Abbott, & Hill, 1999; 
Munoz, Fischetti, & Prather, 2014), the matching procedures were not described (e.g., 
Hasson, 2011; Risner, Vanderhaar, Munoz, & Abati, 2005; St James-Roberts & Singh, 


25 Studies reporting a large difference between the treatment and control group at pre-test were said to be 
excluded as they would not help in distinguishing intervention effects from other effects. The protocol also stated 
that quasi-experimental studies based on one group pre-test and post-test designs or one group post-test-only 
designs should be excluded from the review. 
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2001) or the balance procedures did not produce statistical equivalence (e.g., Gao, 
Hallar, & Hartman, 2014). 

2. Anumber of the school-based intervention programmes included in this review 
presented several studies, involving quasi-experiments as well as RCTs. Some 
examples involve interventions such as the Positive Action Program or the School- 
Wide Positive Behavioural Interventions and Supports (e.g., Lewis et al., 2013; Snyder 
et al., 2010). In both cases, RCTs (e.g., Lewis et al., 2013; Snyder et al., 2010) were 
preceded by quasi-experimental studies (Barrett, Bradshaw, & Lewis-Palmer, 2008; 
Flay & Allred, 2003) . In this context, we decided to keep the strongest study design. 

3. RCTs are regarded to be the most compelling methodological design to test the 
impact of a particular treatment. This type of study has the strengths of isolating 
confounding factors, reducing the likelihood of alternative explanations for observed 
effects (Shadish, Cook, & Campbell, 2002; Sherman, Farrington, Welsh, & Mackenzie, 
2002). We believe that by selecting only these studies we will achieve a more precise 
final estimation of the effect of school-based interventions. 


To offer a broad overview of the research testing the impact of school-based intervention at 
reducing school exclusion, a list of the quasi-experimental studies can be provided on 
request. 


Qualitative studies were excluded from the present review as stated in the published protocol. 


3.2.2 Types of participants 


The present review is focused on the general population of students in primary and 
secondary schools irrespective of nationality, ethnicity, language, and cultural or socio- 
economical background. By targeting primary and secondary schools, participants could 
theoretically be aged from 4 to 18 years of age. 


Reports involving students who presented special education needs, disabilities or learning 
problems but were educated in mainstream schools were included in this review. However, 
reports involving students with serious mental disabilities or those in need of special schools 
were excluded. The rationale for this is that the results of this review are intended to be 
generalisable to mainstream populations of students in non-specialised schools from all the 
included countries. 


Students in college or higher levels of education have been excluded. Their exclusion from 
the review is based on previous evidence suggesting the largest number of exclusions affect 
pupils aged about 10 to 15 (e.g., Liu, 2013; Raush & Skiba, 2004; DfE, 2012). 


3.2.3 Included interventions 


We include interventions defined as school-based: that is, delivered on school premises, or 
supported by schools with at least one component implemented in the school setting. In the 
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present review, we include interventions explicitly aimed at preventing/reducing school 
exclusion or those measuring exclusion as an outcome. 


Interventions in the present review cover a wide range of psychosocial strategies for targeting 
students (e.g., Cook et al., 2014), teachers (e.g., Ialongo, Poduska, Werthamer, & Kellam, 
2001), or the whole school (e.g., Bradshaw, Waasdorp, & Leaf, 2012). Types of intervention 
include, for example, those focused on: 


e instructing students to identify risky behaviours and expanding their alternatives for 
responding appropriately to risks or harms (e.g., social skills training) 

e developing teachers’ skills to improve the quality of their classroom management 
(e.g., reward schemes) 

e cognitive-behavioural treatment, such as anger management, counselling, social 
work, and mentoring programmes; 

e school-wide interventions. 


Since there was no previous review analysing school-based prevention programmes for 
reducing exclusionary discipline, we wanted to include a wide range of school-based 
interventions that could be effective for reducing exclusionary practices. 


3.2.4 Excluded interventions 


We excluded studies where the intervention was not school-based or school supported. Even 
though some of these interventions targeted school students, they were community 
programmes or purely focused on mental health issues without any connection to schools 
(e.g., Henderson & Green, 2014; Schwartz, Rhodes, Spencer, & Grossman, 2013; Wiggins et 
al., 2009). 


We also excluded interventions designed for children or adolescents who have committed a 
crime, that is, specialised interventions aimed at reducing reoffending or reconviction. 
Although suspended students may commit offences, such specialised interventions were 
excluded from the present review because they exceed the strategies used by schools to 
prevent misbehaviour and their levels of complexity make them too specific for a general 
population of students. School-based prevention programmes targeting outcomes related 
only to students’ physical health (e.g., AIDS/ HIV prevention programmes, programmes to 
develop healthy eating programmes) were also excluded. 


3.2.5 Types of outcome measures 
Primary outcomes 


Eligible studies addressed school exclusion as an outcome. As mentioned in the background 
section, school suspension or exclusion is defined as an official disciplinary sanction imposed 
by an authority and consisting of the removal of a child from their normal schooling. This 
removal happens as a reaction to student behaviour that violates the school rules. We 
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included studies testing fixed or permanent, long-term or short-term suspension as well as 
in-school and out-of-school suspensions. 


We excluded studies testing other disciplinary sanctions implemented in schools if they do 
not share the criteria described above. For instance, we excluded disciplinary sanctions such 
as loss of privileges, extra work, break/lunch detention, and after-school detentions. These 
interventions do not involve exclusion from school or exclusion from regular teaching hours, 
and as such they are not covered by this review. 


Secondary outcomes 


For any identified study that reported findings on school exclusion as an outcome, we also 
coded the effects of the intervention on specific behaviour domains, focusing on internalising 
(e.g., inhibition, social withdrawal, anxiety or depression) and externalising (e.g., defiant or 
delinquent behaviours or aggressive behaviours such as bullying) problem behaviour 
(Achenbach & Edelbrock, 1979; Achenbach, 1978; Farrington, 1989) 


By coding secondary outcomes, we aimed to assess the extent to which reductions in problem 
behaviour are a mediator of treatment effects on school exclusion. Indeed, interventions may 
affect exclusion in two different ways. The first is by improving behaviour that might 
otherwise lead to an exclusionary measure. The second possibility is that behaviour stays the 
same, but that the school develops an alternative strategy to deal with the disciplinary 
problems. 


3.2.6 Included literature 


Databases and journals were searched from 1980 onwards with the aim of comprising more 
contemporary interventions or prevention programmes. Eligible studies included both 
published and unpublished book chapters, journal articles, government reports, and Doctoral 
theses. When the same data was published in more than one source (e.g., a book chapter and 
a journal article) we used all the linked manuscripts but the most complete report or the 
report measuring suspension was defined as the main source of data (see Section 3.4.5). That 
way we kept as much information as possible from a specific study but avoided over- 
estimation of effect sizes. In cases where it was not clear if the manuscripts referred to the 
same study, we contacted the main author for further information (e.g., email 
communication with Barnes, Bauza, & Treiber, 2003; Bradshaw et al., 2012; Lewis, Romi, & 
Roache, 2012; Snyder et al., 2010; Sprague, Biglan, Rusby, Gau, & Vincent, 2016). 


3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES 


The electronic searches were conducted between September 1 and December 1, 2015. In 
order to reduce the effect of publication bias, an attempt was made to locate the most 
complete collection of published and unpublished papers. 
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3.3.1 Electronic searches 


Below we list details of the 27 electronic databases searched. As noted above, these databases 
included both published (e.g., ISI web of knowledge, PsycINFO) and unpublished reports 
(e.g., Dissertation Abstracts, EThOS) as well as reports from Latin-American countries (e.g., 
Scientific Electronic Library Online — SciELO). 


Table 2: Electronic searches 
Databases 


1. Australian Education Index (AEl) 

2. British Education Index (BEl) 

The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2- 
SPECTR) 

4. BMJ controlled trials 

5. CBCA Education (Canada) 
6. ClinicalTrial.gov 
i. 
8 


ed 


Criminal Justice Abstracts 
. Cochrane Central Register of Controlled Trials (CENTRAL) 

9. Database of Abstracts of Reviews of Effects (DARE) 

10. Dissertation Abstracts 

11. Educational Resources Information Centre (ERIC) 

12. EThOS (Beta) 

13. EMBASE 

14. Google 

15. Google Scholar 

16. Index to Theses Database 

17. Institute of Education Sciences - What Works Clearinghouse 

18. ISI Web of Knowledge 

19. MEDLINE 

20. The National Dropout Prevention Centre/Network 

21. The Netherlands National Trial Register (NTR) 

22. Open Grey 

23. Psych INFO 

24. Sociological Abstracts 

25. Social Sciences Citation Index (SSCl) 

26. Scientific Electronic Library Online (SciELO). Electronic database collecting scientific production from 
developing countries (Spanish and Portuguese) 

27. World Health Organisation International Clinical Trials Registry (WHO ICTRP) 


For each database, we ran pilot searches including the key terms described in Table 3. Four 
categories of key words were used, including: i) type of study; ii) type of intervention; iii) 
population; and iv) outcomes. The pilot searches were useful to adjust the terms, synonyms and 
wildcards as appropriate. They were also helpful in creating combinations of terms that capture 
relevant sets of studies in each database. 
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Table 3: Key words for searches 


Type of study Interventions Population Outcomes 

Evaluation Disciplinary methods Schoolchildren School exclusion 
Effectiveness Token economy Pupils Suspension 
Intervention Classroom management Children Out-of-school suspension 
Program program/ intervention/ Adolescents In-school suspension 
Programme strategies School-aged children Out-of-school exclusion 
Programme effectiveness School management Student In-school exclusion 
Impact Early interventions Youth Suspended 

Effect School support projects Adolescent Expelled 

Experimental evaluation Skills training Young people Expulsion 
Quasi-experimental Outdoor suspension 
evaluation Stand-down 

RCT Exclusionary discipline 
Random evaluation Discipline 

Efficacy trial 


We kept a record with the date of searches, number of reports found, number of reports 
retrieved, key terms included, synonyms, and wildcards used when appropriate. Further 
details of electronic searches are presented in Section 13. 


3.3.2 Other resources searched 


As planned, we contacted key authors requesting information on primary studies that could 
potentially be integrated in this systematic review and meta-analysis. We also reviewed 
reference lists of previous primary studies or reviews related to the intervention/outcomes 
(e.g., Burrell, Zirbel, & Allen, 2003; Gottfredson, Cook, & Na, 2012; Johnson, 2012; Mytton, 
DiGuiseppi, Gough, Taylor, & Logan, 2006; Wilson, Tanner-Smith, Lipsey, Steinka-Fry, & 
Morrison, 2011). 


3.4 DATA COLLECTION AND ANALYSIS 


3.4.1 Selection of studies 
Eligible studies met the following criteria: 


e Reported results of interventions from 1980 onwards 

e Tested the impact of a school-based intervention on different types of exclusion (e.g., 
in-school, out-of-school, expulsion) 

e Included students from primary and secondary school levels settled in mainstream 
schools 

e Based on an experimental design, where participants are randomly allocated 
treatment or control conditions 

e Reported statistical results for computed an effect size 
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3.4.2 Data extraction and management 


Data extraction was the responsibility of two researchers (AC & SV). Descriptive data of all 
studies potentially includable in the meta-analysis was extracted using the data collection 
instrument presented in Section 12.2. The instrument facilitated the extraction of the 
following information: 


e Bibliographical data (e.g., type of publication, year of publication, name of the 
publication, main author discipline) 

e Ethics (e.g. declaration of conflicts of interest, use of informed consent) 

e Research methods (e.g., type of design, units of randomisation, unit of analysis, 
variables used for matching) 

e Sample selection (e.g., methods to select sample, attrition) 

e Primary outcome coding (e.g., type of exclusion, duration of exclusion) 

e Secondary outcomes coding (e.g., internalising and externalising behaviours, name of 
the instrument used to measure the outcome data) 

e Base-line measurements (e.g., source of data, quantitative measure of the primary 
outcome) 

e Programme delivery (e.g., programme deliverer, training, type of intervention, 
frequency of the intervention) 

e Post-intervention and follow-up measurement (e.g. official records, surveys) 

e Data for calculation on effect sizes 


The same two researchers extracted data for effect-size calculations. The process was carried 
out independently. In general, discrepancies were solved by agreement but when the 
information reported was contentious, we asked for input from the more senior members of 
the team (ME & DF). Details on the data extracted from each included report can be found in 
section 9.1, 9.2 and 9.3 of the present review. 


When data for calculation of effect sizes was incomplete we used two different strategies. 
First, we tried to find more details in other sources (e.g., published protocols or reports). 
Secondly, the lead researcher or members of the research team were contacted regarding the 
additional data needed. 


Endnote X7 software was used to manage references, citations and documents. Data 
extracted to characterise studies was inputted in STATA v.13 in order to produce 
inferential/descriptive statistics. Effect sizes were inputted in Version 3.0 of the 
Comprehensive Meta-Analysis software. 


3.4.3 Strategy to test inter-rate reliability 


To check code consistency across studies, or inter-rate reliability, we use Cohen’s Kappa 
coefficient (Cohen, 1960). In the event of two coders making inclusion/exclusion decisions, 
Cohen’s Kappa coefficient computes a standardised index across studies based on cross- 
tabulated ratings. The index is given by the difference between the observed percentage of 
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agreement in ratings across studies (P,), and the probability of expected agreement due to 
chance (P-), divided by 1-P:. 


Cohen’s Kappa was calculated based on the following formula: 


k = PoxPe (1) 


1—Pe 


To a good approximation, we calculated the standard deviation of Cohen’s Kappa, following 


the expression (Cohen, 1960): 
—s Po(1—Po) 
SD, = | Cae (2) 


3.4.4 Assessment of risk of bias in included studies 


We planned to assess the risk of bias in included studies by using two different instruments. 
In the case of RCTs, we intended to use the Cochrane Collaboration risk of bias tool (Higgins 
& Green, 2011). To analyse risk of bias of studies involving quasi-experimental designs we 
planned to use the ACROBAT-NRSI, another Cochrane Risk of Bias Assessment Tool for 
Non-Randomised Studies (Stern, Higgins, & Reeves, 2014). Both instruments were supposed 
to assist the research team in evaluating the external validity of the included reports. 


At the end of January 2016, we began the assessment with the originally proposed 
instruments. We soon realised that the instruments and their categories seemed more suited 
to medical trials than school-based experiments. Therefore, possible alternatives were 
explored. In consultation with the coordinating group editor, we selected the EPOC risk of 
bias tool suggested in the methods section of the Campbell Collaboration website (see the 
tool in section 12.3). The instrument proposes the following eight criteria for the assessment 
of quality bias: 


e Sequence generation 

e Allocation concealment 

e Baseline outcome equivalence 

e Baseline characteristics equivalence 
e Incomplete outcome data 

e Blinding of outcome assessment 

e Protection against contamination 

e Selective outcome reporting 


Each of these domains was judged on a 3-point scale (i.e., low risk, high risk, unclear risk). 
EPOC tool provides guidance and examples for each domain that facilitate the decision of 
assigning low, high or unclear risk. Two members of the team performed the assessment of 
risk of bias (AS & SV). Assessment of bias was performed independently and the final results 
represent the agreement of both evaluators. 
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3.4.5 Criteria for determination of independent findings 


Since violations of the assumptions of independence in meta-analysis would lead to incorrect 
estimates of the variance for pooled effect sizes (Higgins & Green, 2011b; Romano & 
Kromrey, 2009), we used some strategies to deal with dependency in the data extracted from 
primary studies . 


First, since we included book chapters, journal articles, government reports, and academic 
PhD theses, we anticipated the case where the same results would be published in more than 
one source (e.g., a book chapter and a journal article). The protocol stated that in those cases 
we would code only one outcome (e.g., the most complete, or the most outdated). In practice, 
we excluded 11 reports whose results were reported in more than one publication. They are 
grouped in four cases: 


e The study did not report enough statistical data for effect size calculation. For 
instance, Vincent, Sprague, Pavel, Tobin, & Gau, (2015) did not report enough data 
for meta-calculation. Although the main author was contacted, we were not able to 
access more details. For that reason we decided to include Sprague, Biglan, Rusby, 
Gau, & Vincent, (2016). This latter case used the same data but reported enough 
information for meta-analysis. There was no overlap. 

e RCT data was merged with quasi-experimental data. For instance, Allen & Philliber, 
(2001) merged the RCT sample with another sample of students. As a result, the 
study was not an RCT anymore (For further details see Allen & Philliber 2001, p. 641). 

e The same results were reported in more than one manuscript. That was the case of 
Panayiotopoulos & Kerfoot. The results of the study were reported in two papers 
(2004 and 2007). Results were identical in both publications (same dataset, same 
analysis and same outcomes). There was no overlap. 

e The study did not report the outcome measured (i.e., suspension). A case in point is 
Arter, (2007). The author reported the results of her thesis in a journal. The journal 
article did not describe the outcome suspension, probably because no effect was 
found. We included the thesis since it reported all the outcomes measured. 


Based on that, those 11 manuscripts were not included in the meta-analysis. There was no 
overlap among them, consequently, no dependence in the outcomes was observed. 


Secondly, included studies reported multiple time points, for instance, multiple follow-up 
measures. The inclusion of multiple follow-ups would create statistical dependence because 
the different measures are based on the same subjects (i.e., correlated with each other). We 
calculated effect sizes separately for those studies reporting short-term and long-term follow- 
up measures. We also corrected variances estimation (see 3.4.8). 
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3.4.6 Measures of treatment effect 


We use Standardised Means Differences (SMD or Cohen’s d) to measure the treatment 
effects of the school-based interventions included in the review. The decision to use this 
specific effect size is based on the fact that most of the included manuscripts report results 
where measurements are expressed in continuous scales (see section 9.3). The standardised 
mean effect size for a non-clustered study is given by 
ee 


d 
Sp ’ 


(3) 


where X, and X, represent the experimental and control group means, respectively, 5, is the 
pooled sample standard deviation given by 


_ | Men 1)SE+Me-1)5E 
aan cei eey (4) 
where n, and n, are the sample size in each group, and S2 and S? are the experimental and 
control group standard deviation, respectively. 


3.4.7. Issues with the unit of analysis 


In the present review, we anticipated the inclusion of primary studies involving individually 
randomised as well as cluster-randomised unit (e.g., schools or classrooms). One key issue 
emerges when meta-analyses include cluster-randomised studies: participants nested in the 
same cluster tend to be more similar to one another (as measured by the intra class 
correlation — ICC). Furthermore, when units of randomisation are clusters instead of 
individuals, we need to deal with the fact that the data presents different levels of variation 
(i.e., within clusters variation, between clusters variation and the total variance). This issue 
needs to be taken into account when computing effect size estimates. When this correlation is 
not accounted for, standard errors, confidence intervals and p-values will tend to be too 
small. These conditions affect the meta-analysis in two different ways. Firstly, the primary 
trial gets a mistakenly large weight. Secondly, the pooled result produces estimated effect 
sizes with an overly small standard error (Borenstein, Hedges, Higgins, & Rothstein, 2009; 
Higgins et al., 2011). 


For the case of clustered data with dichotomous outcome measures (e.g., odds ratios), we 
followed the strategy proposed by the Cochrane Handbook for Systematic Reviews of 
Interventions (Higgins & Green, 2011), which corrects standard errors of effect sizes. The 
handbook suggests that the effective sample size in a cluster-randomised trial can be 
obtained by dividing the original sample size by the design effect, which is calculated via 


1+(M—1)x ICC. (5) 
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In this equation, M is the average cluster size (units per cluster) and ICC is the intra-cluster 
correlation coefficient. Once we were able to identify the design effect, the squared root of the 
design effect could be multiplied by the original standard error of the log Odds Ratio. Since 
ICC is rarely reported in primary studies, we have assumed a value of .05, based on the 
review of multiple meta-analyses testing similar populations, produced by Ahn, Myers, & Jin, 
(2012). 


In the case of clustered studies with continuous outcomes (e.g., school level means and 
standard deviations), we followed the strategy suggested by Hedges (2007) and Spier et al. 
(2013). Effect sizes were computed using dr. assuming equal cluster sample size: 


_ [Xe-xX§ | __ 2(n-1)p 
adr = | = | 1 N-2 (6) 


In this equation X£ and X§ represent the overall means of the experimental and control 


group and Sr is the total sample standard deviation estimated from the pooled sample 
standard deviation across the experimental and the control group. Rho (p) is the notation 
used to represent the intra class correlation. N is the total sample size and the sample size of 
the clusters is represented by n. Based on the characteristics of our data and following Spier 
et al., (2013) we assume equal cluster size in our calculations.2° When the clusters have 
different sizes, we will take a conservative approach, including the smallest cluster size in our 
calculation. 


The variance of the effect size will be calculated by 


NE+NCE 


Vidro} = (SAE) A+ (- 1p) + 


d2 (SoBe eGo) tah tad 22m) 
He 2(N-2)[(N-2)-2(n—1)p] 


(7) 


In this equation, N# and N¢ represent the experimental and control group sample across 
clusters. As suggested by Higgins & Green, (2011), in the event that the value of p is not 
reported, analysts are advised to assume a reasonable value based in previous studies with 
similar population (Ahn et al., 2012). As detailed previously, we have assumed a value of (—¢)= 
.05. 

3.4.8 Dealing with missing data 


In those reports where key statistical information was missing, we attempted to obtain data 
from principal investigators. When that was not possible, the study was excluded from 
calculations of effect sizes. All the studies excluded for this reason have been identified and 
systematically reported in section 4.1.3. 


26 The assumption of equal sample size seems to be a good approximation for the calculation of effect sizes. 
Hedges, (2007) asserts that effect size calculation based on equal and unequal cluster sizes are not substantially 
different. 
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3.4.9 Time points within a study 


In the present review, many of our selected studies involved repeated measures of the 
outcome exclusion. In fact, manuscripts reported measures of exclusion at baseline and post 
treatment (e.g., Hawkins, Doueck, & Lishner, 1988) or post treatment and follow-up (e.g., 
Farrell, Meyer, & White, 2001). In these cases, we have calculated a synthesis index or effect 
size of the difference, representing the change between those different measures (i.e., time 
points). Consequently, the change in exclusion from the baseline is computed by subtracting 
the means (X) as follows: 


Yaigg = X2—-X (8) 


One issue arises when pursuing this strategy. Because measures at baseline and post 
treatment are positively correlated, the calculation of the variance must be corrected. If we 
avoid the correction, assuming the two measures to be independent (correlation equals zero), 
we could be overestimating the variance and underestimating the precision of the difference 
(Borenstein et al., 2009). For a fair approximation of the value of the variance, we would 
need to know the correlation between the pre- and post-measures (covariation). However, 
the covariance is not usually reported in primary research (and this was commonly the case 
in our set of included studies). Consequently, we proceed to assume a value for that 
correlation. After checking previous meta-analysis of similar populations, testing school- 
based interventions with estimates for the stability of serious problem behaviours (e.g., 
Farrington & Ttofi, 2009; Wilson et al., 2011), we concluded that the value of the pre-post 
correlation should be assumed to be equal to .75. We then calculated the variance of the 
difference by using equation 9 below (Borenstein et al., 2009), where V, and V2 represent the 
variances of the original point estimates and r represents the pre-post correlation value: 


VYaigp = Vi t+ Vo — 2r/V, JVo. (9) 


As suggested by Higgins & Green, (2011), we undertook sensitivity analyses to determine 
whether the overall result of the analysis is robust in the use of imputed correlation 
coefficient. 


3.4.10 Assessment of heterogeneity 


We report weighted mean effect sizes, under a random model using 95% confidence intervals 
and accompanied with graphical representation (i.e., forest plots). For investigating 
heterogeneity, we use the estimates suggested by Borenstein et al. (2009), specifically; Tau- 
squared, Q-statistic and I2. 


Tau-squared, or the difference between the total variance or variance observed and the 
within-studies variance, will be estimated and reported. 
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The final calculation of the Q-statistic includes reporting its value, degrees of freedom and p- 
values. Significant p-values provide evidence of heterogeneity in intervention effects. 


Bearing in mind that Q can appear distorted when the number of studies meta-analysed is 
small (Higgins, Thompson, Deeks, & Altman, 2003), we also report F°. I? “is the proportion of 
observed dispersion that is real rather than spurious” (Borenstein et al., 2009). High 
percentages will be interpreted as an indication of high heterogeneity, meaning that the 
study-to-study dispersion is due to real differences in true effect size and not attributable to 
random error. 


3.4.11 Data synthesis 


Since our review has a wide scope, we use the random effect inverse variance weighted 
models for meta-analytical calculations. The random effect model is the most appropriate 
when effect sizes are not homogeneous or consistently coming from a single population 
(Borenstein et al., 2009). Under a random effects model the variance includes the original 
(within-studies) variance plus the between-studies variance, Tau2. 


Following Chandler, Churchill, Higgins, Lasserson, & Tovey, (2013), effect sizes will be coded 
such that a positive effect will reflect the outcomes favouring the treatment group. To 
illustrate our analysis, we provide summary forest plots displaying the estimated effect sizes 
along with their 95% confidence intervals. 


3.4.12 Subgroup analysis and investigation of heterogeneity 


In the present review, we use moderator analysis involving categorical variables estimating 
models analogous to ANOVA. Analyses are run under a random-effect model assuming 
separate variance components for each group. Meta regression has been run in order to 
explore heterogeneity. 


3.4.13 Outliers 


The distribution of SMD effect sizes was examined to determine the presence of outliers. 
Following Lipsey & Wilson, (2001), outliers were defined as those values which are more 
than two standard deviations from the overall mean of effect sizes. One outlier was detected 
(i.e., Collier, 2002) and it was windsorised to the next closest value (Lipsey & Wilson, 2001). 


3.4.14 Sensitivity analysis 


Since the present meta-analysis involved a wide range of decisions, we conducted sensitivity 
analysis to test the robustness of these decisions (Higgins & Green, 2011). Specifically, we ran 
sensitivity analysis for the pre-post correlations (i.e., covariance) assumed to be .75. We re- 
ran the analysis using a correlation equal .50. As expected there was no change in the effect 
sizes and no relevant difference in standard errors. We also ran sensitivity analysis testing 
the impact of the outlier and the impact of the windsorisation (see section 4.7). 
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As stated in our protocol, we ran sensitivity analysis testing differences between published 
and unpublished reports. 


3.4.15 Duration of follow-up 


Included studies reported multiple time points data, for instance, multiple follow-up 
measures. Since the inclusion of multiple follow-ups would create statistical dependence due 
to the different measures based on the same subjects (i.e., correlated with each other), we 
calculate effect sizes separately for those studies reporting short-term (i.e., post-treatment) 
and long-term (i.e., follow-up) measures. 
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4. Results 


4.1 DESCRIPTION OF STUDIES 


4.1.1 Results of the searches 


We attempted to identify and retrieve the body of published and unpublished studies that 
met our inclusion criteria. Figure 1 shows a PRISMA flow diagram describing the results of 
our searches. 


At the beginning, different combination of terms produced a total of 42,749 references from 
different electronic databases, of which we kept 1,474 relevant hits. 


The 1,474 hits were selected based on screening the title, abstract and key words. We targeted 
studies broadly defined as evaluations, testing the impact of interventions on school 
exclusion. 


As originally planned, we complemented electronic searches for papers with two extra 
strategies: i) reviewing lists of references from retrieved manuscripts, and ii) communication 
with key authors. Based on these steps, an additional 22 manuscripts were added to our 
sample. The full list of studies and their references were imported into Endnote X7. After 
removing duplicates, a total of 516 unique manuscripts were saved for further assessment. 


Efforts were made to retrieve the full text copies of all 516 selected manuscripts. Notably, a 
high percentage of them corresponded to unpublished reports (48.6%), mainly PhD theses 
from American universities and Technical or Governmental reports. In the end, we were able 
to retrieve almost all full text manuscripts. Only eight studies out of 516 were categorised as 
“studies awaiting classification” and they are reported in Table 6. 


The next round of evaluation was based on reading the information available in abstracts, 
methods and results sections using the eligibility checklist (see Section 12.1). The checklist 
included the basic criteria for inclusion in this systematic review. We concluded the second 
round of evaluation with 471 manuscripts excluded for reasons laid out in section 4.1.3. 
Thirty-seven papers presented enough statistical data for inclusion in our meta-analysis. 
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Figure 1: PRISMA flow’ chart of searches 


Total of references based on combination of terms 
(n = 42,749) 


Additional records identified through other 


Records identified through database 
sources (n =22) 


searching (n =1,474) 


Identification 


Records after duplicates removed 
(n= 516) 


Studies awaiting 
classification 
(n =8) 


Records screened 
(n =516) 


Full-text articles assessed for Full-text articles excluded, 


eligibility (n = 508) with reasons (n = 471) 


Eligibility 


Studies included in meta- 


analysis (n =37) 


4.1.2 Inter-rater reliability 


Two trained researchers (AC & SV) independently assessed the 516 pre-selected manuscripts 
for inclusion or exclusion. We calculated Cohen’s Kappa for testing inter-rater reliability 
(Cohen, 1960; Sim & Wright, 2005). The value of Kappa ranges between 0 and 1, where a 
value of 1 represents perfect agreement between the two raters and a value of 0 indicates no 
more agreement than that expected by chance. We obtained a Cohen’s Kappa= .76; SD=.81; 
SE=.036, reflecting a high level of agreement between coders. After calculating the 
agreement between coders, they went through the papers where they found disagreement. 


Differences were solved by further analysis and discussion. When needed, a more senior 


member of the team was consulted. 


27 The presented flow chart has been adapted from Moher, Liberati, Tetzlaff, & Altman, (2009) 
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4.1.3 Excluded studies 


Following our published protocol, we excluded a total of 471 manuscripts. Table 4 
summarises the reasons for the exclusion of each report. References of the excluded papers 
are available in section 7.2. 


Table 4: Synthesis of the reasons for the exclusion of 471 papers 


Reason for exclusion k % 
Outcome measure was absent 52 11.0 
Type of intervention 53 11.2 
Methodological design 339 72.0 
Participants 5 11 
Time span 5 1.1 
Pilot study 1 0.2 
Reports based on the same data 11 2.3 
Not enough data for meta-calculations 5 14 
Total 471 100 


Outcome. We excluded 52 reports (11%) because they did not present a suitable measure of 
school exclusion. In some specific cases the primary outcome was not reported (e.g., Gage, 
Sugai, Lewis, & Brzozowy, 2015; Webster-Stratton, Rinaldi, & Reid, 2011) or it was reported 
in a composite measure along with other disciplinary measures which did not involve any 
type of exclusion from school (e.g., De Blank, 2009; Wright, Offord, John, Duku, & DeWit, 
2005). Since it was not possible to isolate our primary outcome we excluded those reports. In 
other exceptional cases exclusion was measured as a predictor instead of an outcome (e.g., 
Rosenbaum, 2012). 


Type of intervention. Following our protocol, we excluded 53 (11.2%) studies because the 
tested intervention was not delivered in schools, supported by schools or with at least one 
component implemented in school settings. Consequently, we excluded community 
programmes when they had no alliance with a school (.e.g., Henderson & Green, 2014; 
Wiggins et al., 2009). Alternative schools for high-risk students (e.g., Rhea, 2010) and 
intervention in special schools (e.g., Kutash, Duchnowski, Green, & Ferron, 2013) were also 
excluded. Since our protocol stated that targeted intervention must be an “alternative to 
school exclusion”, we dismissed studies testing for instance the impact of restorative justice 
in the context of an in-school exclusion programme (e.g., Brown-Kersey, 2011), or conflict 
resolution in the context of an in-school exclusion programme (e.g., Devlin, 2006). In both 
cases, the tested intervention was delivered in addition to exclusion more than being an 
alternative to it. 


Methodological design. 339 (72%) studies were excluded because they did not satisfy the 


methodological characteristics defined in the protocol. We excluded studies lacking a control 
group (e.g., McDaniel-Campbell, 2011; Nocera, Whitbread, & Nocera, 2014) and those 
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studies where the control group was not equivalent in demographics and risk factor variables 
(e.g., Kilian, Fish, & Maniago, 2006; May, Stokes, Oliver, & McClure, 2015). 


During the searches, we kept 28 manuscripts which corresponded to literature reviews (e.g., 
Fronius, Persson, Guckenburg, Hurley, & Petrosino, 2016; Gonzalez, 2012), systematic 
reviews (e.g., Wolgemuth, Cobb, & Dugan, 2007) or meta-analysis (e.g., Noltemeyer & Ward, 
2015) related to school exclusion or behavioural problems in schools. These types of 
manuscripts were initially retained on the understanding that they could be a source for 
identifying extra primary research reports. All 28 of these studies were excluded in the 
second round of assessment once we had checked their citation lists. 


We identified nine manuscripts evaluating the impact of obligatory use of school uniform on 
levels of school exclusion (i.e., Draa, 2005; Gentile & Imberman, 2012; Gouge, 2011; 
Johnson, 2010; Samuels & Bishop, 2003; Shimizu & Peterson, 2000; Stevenson III & 
Brooks II, 1999; Vaughan, 2001; Washington-Labat & Ginn, 2003). Even if they represented 
a particular type of intervention that could be interesting to meta-analyse and compare, none 
of these eight interventions presented the research design targeted by our review. For that 
reason, they were excluded. 


Finally, under the method design criteria we excluded a number of qualitative studies 
(Huston, 1999; Maguire, Macrae, & Milbourne, 2003; Rose, 2008) and case studies (e.g., 
Navarro, Aguilar, Aguilar, Alcalde, & Marchena, 2007) since none of them contributed with 
statistical data for meta-analysis. 


Participants. Five reports (1.1%) presented data focused on students with special needs (e.g., 
Cramer, 1990), or young offenders (e.g., Parkes, 2008). Since these participants were not 
targeted in our protocol, all these reports were excluded from our analysis. 


Time span. Five studies (1.1%) were excluded because they were published before 1980 (i.e., 
Rogers, 1972) or because they reported the evaluation of an intervention implemented before 
1980 (i.e., Feldi, 1980; Fisher, 1980; Herzog, 1980; Safer, Heaton, & Parker, 1981). 


Reports based on the same dataset. Finally, we excluded 11 reports (2.3%) because they 
presented additional results based on the same data reported elsewhere (e.g., Allen & 
Philliber, 2001; Arter, 2007; Panayiotopoulos & Kerfoot, 2007; Vincent, Sprague, Pavel, 
Tobin, & Gau, 2015). In these cases, we kept the most complete report and the additional 
manuscripts assisted in a better understanding of the included research; they were however 
defined as excluded. 


Pilot study. We excluded a single report presenting data from a pilot study (Bonell et al., 
2015). In page 12 of the cited report, the author states that the aim of the pilot was “to 
evaluate feasibility and acceptability of the intervention and trial methods, and not to 
estimate intervention effects.” For those reasons, even though it was reporting results, the 


52 The Campbell Collaboration | www.campbellcollaboration.org 


study was excluded. We kept references to the report in the category of ongoing research (see 
section 4.1.4). 


Not enough data for meta-calculations. Five reports (1.1%) were excluded because they did 
not present enough data for calculation of effect sizes (e.g., Grinage, 2005). 


4.1.4 On-going studies 


We identified the protocol of four ongoing studies whose outcomes had not been published at 
the end of the searches in December 2015. As observed in Table 5, all of them are cluster 
randomised control trials testing the impact of school-based interventions and measuring 
school exclusion as an outcome. 


Table 5: On-going studies 


Author Design Sample Outcome of interest Intervention 

Acosta Cluster-RCT Unclear (US) Suspension or The Restorative Practices 

(2015) expulsion Intervention (RPI) 

Bonell et al. Cluster-RCT 40 schools Temporary and INCLUSIVE (combines 

(2014) (UK) permanent school changes to the school 
exclusion environment, promotion of 


social and emotional skills 
and restorative practices) 


Philliber Cluster-RCT 6 schools In- and out-of-school School-wide positive 
(2015) (US) suspension behavioural interventions and 
supports (SWPBIS) 
Eiraldi (2014)  Cluster-RCT 12 school School suspension Teen Outreach Programme 
US in Kansas City (replication 


4.1.5 Studies awaiting classification 


We were unable to classify eight studies. We selected them based on title, abstract and key 
words but we have not been able to locate the full text copies. A list of studies that could be 
potentially included in a future updated version of this review is given in Table 6. 
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Table 6: Details of studies awaiting classification 


Author Type of Design Sample Outcomes Intervention 
publication 
Allen Report Pre-post design. It 12 seventh Disciplinary referrals Positive approach to 
(1981) is unclear if the grade Corporal punishment discipline (PAD) is a 
study uses random _ teachers School suspensions system of classroom 
allocation of management, 
participants. incorporating 
counselling, problem- 
solving, and time-out 
centres. 
Forbes Thesis Pre-post design. It 900 students School infractions Social skills training 
(1996) is unclear if the (grades six to Out-of-school 
study uses random _ eight) suspension 
allocation of 
participants. 
Foster Book No information African No information Social Skills 
(2011) American Curriculum 
boys in 
elementary 
classrooms 
Gaines & Conference __ Intervention is Middle and School absences School Probation 
Schram proceedings given to some high school Suspension/expulsion Officer Programme is 
(2005) students, but not to students Disciplinary actions aimed at identifying 
others. Unclear if it Attendance juveniles who may be 
is RCT. Grades at-risk of engaging in 
delinquent behaviour 
Gallegos Book Review of Unclear Suspensions Not given 
(1998) interventions Expulsion 
Dropout school 
Neise Thesis Two treatment 37 middle The Devereaux Group counselling 
(1983) groups and one school Adolescent Behaviour methods that used 
control group. students Rating Scale (DABRS) interpersonal problem 
Unclear if they The Behaviour Rating solving strategies 
were randomly Scale (BRS) versus non-directive 
allocated in Seventh hour counselling 
conditions In-school suspension 
Out-of-school 
suspension 
Norris Conference — Evaluation Unclear Suspension Restorative justice 
(2009) proceedings Expulsion 
Spillman Thesis Unclear Ninth grade Achievement Interdisciplinary 
(1993) students Motivation teaming and parent 
Attendance contacts 
Suspension rate 
4.1.6 Included studies 


Thirty-seven studies reporting 38 interventions’ effect sizes were included in this meta- 


analysis. As we mention before, all of them were randomised controlled trials. In 23 studies 
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the control group received no-treatment (62.2%); six studies reported controls receiving 
intervention or business as usual (16.2%), four experiments offered a placebo to the control 
group (10.8%) and four studies allocated controls in a waiting list (10.8%). 


Following sections provide a general overview of 37 included studies. Data is organised by 
characteristics of the included studies, characteristics of participants and characteristics of 
the delivered interventions. 


Characteristics of the included studies 


We included studies presenting interventions (Mdate=2003; SD=9.5) carried out and reported 
between 1980 and December 1, 2015 when we finished our searches. Exceptionally the review 
involves three manuscripts published in 2016. In the first case, Obsuth et al., (2016), the 
registered protocol of the study had been identified in our electronic searches and we were 
waiting for the published version, which was released in March 2016. The second case 
involves two different papers: Okonofua, Pauneskua, & Walton, (2016) and Sprague, Biglan, 
Rusby, Gau, & Vincent, (2016). Both studies were sent to us from key authors in the field. As 
they were recent studies, matching our inclusion criteria, we decided to retain them in our 
analysis. No other study was sent to us after the end of the searches in December 2015. 
Figure 10.1 (in appendix) presents the distribution of studies per year. Results show that the 
number of RCTs has increased over the specific years involved in this report. 


As shown in Table 7, we included published and unpublished reports in almost equal 
proportion (51% published versus 49% of unpublished reports). All of them were written in 
English, and represent studies implemented in the United States and the United Kingdom 
only. Although we explored global databases, in particular databases from Latin-American 
countries we were not able to find studies conducted in other locations. 


We coded data on the main author’s discipline. Our data shows that more than 60% of 
studies testing interventions intended to reduce exclusion have been carried out by 
researchers in the fields of Education and Psychology. 


Interestingly, only seven studies (36.8%) of experimental evaluations published in peer- 
reviewed journals disclosed a personal or organisational Conflict of Interest (Col). This 
percentage is coherent with the findings of Eisner, Humphreys, Wilson, & Gardner, (2015) 
who found limited attention to full CoI disclosure in the evaluation of psychosocial 
interventions. They argue that even if “transparency about Col in itself does not necessarily 
improve the quality of research, and researchers with a CoI should not be presumed to 
conduct less valid scholarship, transparency is needed for readers, to assess the study 
findings and their particular context” (Eisner et al., 2015, p. 10). 


In addition to the presence/absence of CoI statements, we evaluated studies on their 
potential conflict of financial interest (CoFT) by using a scale developed by Eisner & 
Humphreys, (2012). It is a trichotomous scale that helps to identify three levels of conflict. 
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The three categories in the scale are defined as follows: i) Unlikely conflict of interest: none of 
the study authors are programme developers or licence holders; ii) Possible: a study author is 
a programme developer or collaborator with a programme developer AND the programme is 
not (yet) commercially available OR the business model is ‘not-for-profit’; or iii) Likely: study 
author, is a programme developer or collaborator with a programme developer AND 
programme is commercially available AND business model is ‘for-profit’. For details on the 
instrument see section 12.4. 


We found 18 studies (48.6%) where the CoFI was defined as ‘unlikely’. Essentially, in this set 
of studies, none of the programme evaluators were involved (i.e., directly or as a collaborator) 
in the development of the intervention or were licence holders. However, we found 13 studies 
(35.1%) where we assessed a ‘possible’ CoFI. In those cases, the evaluator was a programme 
developer/deliverer or a previous collaborator with a programme developer; the programme 
was not commercially available, or the business model was defined as ‘not-for-profit’. Finally, 
only two studies (5.4%) in our evaluation transparently declared information that allowed us 
to classify them as ‘likely’ to present a potential financial conflict of interest. In both cases the 
authors reported that one of the members of the evaluation team was related to the holder of 
a licensed programme evaluated. The evaluation of each study is reported in section 9.2. 


In terms of methodological design, all included studies were randomised controlled trials. 
While 70.3% of our studies randomised individuals, almost 30% (11 studies) randomised 
clusters of students, that is, entire schools (i.e., Bradshaw et al., 2012; Cornell, Allen, & Fan, 
2012; Lewis et al., 2013; Obsuth et al., 2016; Snyder et al., 2010; Sprague et al., 2016; Ward & 
Gersten, 2013; Wyman et al., 2010) or classrooms within schools (i.e., Brett, 1993; Farrell, 
Meyer, & White, 2010; Ialongo, Poduska, Werthamer, & Kellam, 2001). Where necessary, we 
corrected data in clusters in order to combine it with individual level data. For further details 
on corrections, see section 9.3. 


The most frequent analysis of the data was differences in means such as chi-squared, 
ANOVA, ANCOVA or MANOVA (59.4%) followed by regression (21.6%), and with a minority 
of the studies running Multi-level analysis (10.8%). 


Additionally, measures of disciplinary exclusion were mainly based on official records (81%) 
provided by schools or other official institutions. Table 7 offers a general description of 
studies’ sample size. The average sample size was Msize=1,168 (SD=3107.3) participants. But 
this average should be cautiously interpreted since the included studies range widely from 20 
to 13,498 participants. This is an important issue for the statistical power of the calculated 
impacts. 
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Table 7: Characteristics of studies included in meta-analysis 
Study characteristics 


Publication year (range) 1980 -2016 

Type of publication n % 
Journal articles (published) 19 51.3 
PhD thesis, technical report (unpublished) 18 48.6 
Language n % 
English 37 100 
Country of the sample n % 
United States 33 89.1 
United Kingdom 3 8.1 
Unclear 1 2.7 
Author’s main discipline n % 
Education 13 35.3 
Social work 1 2.7 
Psychology 12 32.3 
Criminal justice 1 2.7 
Psychiatry/medicine 6 16.2 
Econometrist-economics 2 5.4 
Not reported 2 5.4 
Declared conflict of Interest in published studies (N=19) n % 
Yes 7 36.8 
No 12 63.2 
Conflict of financial Interest n % 
Unlikely 18 48.6 
Possibly 13 35.1 
Likely 2 5.4 
Not enough information 4 10.8 
Unit of randomisation n % 
Individuals 26 70.3 
School clusters 8 21.6 
Classroom clusters 3 8.1 
Statistical analysis n % 
Multilevel modelling 4 10.8 
Differences in means (MANOVA, X2, ANOVA, ANCOVA) 22 59.4 
Regression 8 21.6 
Frequencies 3 8.1 
Exclusion measurement n % 
Self-report 2 5.4 
Teacher-report 3 8.1 
School records (official records) 30 81.1 
Unknown 2 5.4 
Effective2® sample size n % 
< 300 24 64.8 
Between 300 and 800 7 18.9 
> 900 6 16.2 


28 The effective sample size refers to the number of unique students participating in each included randomised 
controlled trial, irrespective if they were part of clusters or not. 
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Characteristics of participants in included studies 


As observed in Table 8, in the present meta-analysis, sampled students most frequently come 
from the higher school grades (i.e., middle and high schools), with a Mage=12.9; SD= 2.8. 


In addition, data shows that students were nested in schools with a high percentage of Black 
and Latino students and, subsequently, with a low percentage of White-Caucasian peers 
(24.6%). In fact, five of the included studies are based in schools where a 100% of the pupils 
were Black (i.e., Barnes, Bauza, & Treiber, 2003; Brett, 1993; Collier, 2002; Mack, 2001; 
Reese, Murphy, & Filipezak, 1981). 


As stated by Hobbs & Vignoles, (2010) and Sutherland, Ilie, & Vignoles, (2015), in this report 
the percentage of students with access to free school meals was understood as an index of the 
vulnerability of the population. Across our selected studies the pupils receiving meals funded 
by the school was predominant (Mgm=66.2; SD=23.9%). 


Table 8: Characteristics of participants included in meta-analysis 


Population’s characteristics 


mean standard deviation 
Study average age 12.9 2.8 
Grade frequency % 
Elementary school 11 29.7 
Middle school 16 43.2 
High school 8 21.6 
Mixture of levels 2 5.4 
Ethnicity mean standard deviation 
% Black or Afro-American 54.1 37.2 
% White 24.6 27.9 
% Latino 20.2 25.5 
Free school meals mean standard deviation 
66.2 23.9 


Note. Means and standard deviations are calculated at study level. 
Intervention characteristics: dosage, delivery and targeted change 


The present review includes a wide range of school-based interventions covering, for 
instance, strategies such as counselling, mentoring, community services, classroom 
management, after-school academic support, and school-wide strategies, among others. 
Taking a broad view, 27 % of the interventions were focused on a change at the level of the 
school or teacher, and a 73% anticipating that a change in pupils’ skills/behaviours could 
affect suspension/exclusion rates. 


As shown in Table 9, school staff (32.4%) or school staff assisted by external facilitators 
(24.3%) delivered most of the interventions. This means that schools interested in reducing 


58 The Campbell Collaboration | www.campbellcollaboration.org 


exclusion typically put resources into the intervention process. Most of those delivering the 
interventions were school psychologists/counsellors (32.2%) and, in two particular cases 
(5.4%), the intervention was delivered by a police or probation officers, which seems to be an 
increasing trend in the United States (e.g., Gottfredson et al., 2012; Hirschfield, 2008). 


We coded data on the role of the evaluator. We identified four potential approaches taken by 
evaluators; i) delivering the programme, meaning that the researcher implemented the 
intervention as well as acting as evaluator (e.g., Harding, 2011); ii) designing the programme, 
meaning that the evaluator defined or took part in the defining the theoretical base, aims and 
activities of the intervention (e.g., Cornell et al., 2012), iii) both, designing and delivering the 
programme (Mack, 2001) and iv) independent evaluation, referring to those researchers or 
research teams where any member was involved in any stage of the designed or delivery (e.g., 
Bradshaw et al., 2012; Dynarski et al., 2003). A high percentage of interventions were 
designed and/or delivered by the same researcher who evaluated the impact of the 
intervention. Independent evaluators conducted 48.6% of the RCTs included in this review. 


A high percentage of the interventions were curricular (67.6%). For instance, social skills 
trainings or anger management training were based on a pre-designed curriculum with 
detailed aims and activities for each session. In the present review, non-curricular 
interventions are those targeting school-wide change or those focused on counselling and 
individual therapies. 


On average, included interventions lasted M=20; SD=11.5 weeks. 37.8% of the interventions 
lasted less than 12 weeks and an equal percentage were delivered over more than 24 weeks. 
In general, school-wide interventions lasted longer in our sample (i.e., 35 weeks or more). 
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Table 9: Characteristics of interventions included in meta-analysis 


Intervention characteristics Frequency % 
At school level 10 27.0 
At students’ level 27 73.0 
Programme delivery 

External facilitators 9 24.3 
School facilitators 12 32.4 
School facilitators plus external facilitators 12 32.4 
Missing 4 10.8 


Who delivered programme? 


Social worker 2 5.4 
Psychologist/counsellor 12 32.2 
Teacher 9g 24.3 
Police/probation officer 2 5.4 
Trained community agent 7 18.9 
Not clearly stated 5 13.5 
Role of the evaluator 

Deliver the programme 4 10.8 
Design the programme 8 21.6 
Delivery and design 3 7.8 
Independent 18 48.6 
Unknown 4 10.8 
Is the programme curricular? 

Yes 25 67.6 
No 12 32.4 
Duration in weeks 

Less than 12 weeks 14 37.8 
Between 13 and 24 weeks 4 10.8 
More than 24 weeks 14 37.8 
Unknown 5 13.5 
Duration of intervention m sd 
Hours per week (n=25) 1.78 2.09 
Number of weeks 20.4 11.5 


Note. For specific details regarding each included study, see section 9.1 in appendix. 


4.1.7. Description of the interventions 


Table 10 provides a description of the included school-based programmes targeting school 
exclusion as a primary or secondary outcome. In some specific cases this grouping could be 
restrictive, because some of the interventions involve multiple components, however we have 
attempted to create an exhaustive list of categories. 
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Table 10: Types of intervention programmes 


Programme Number of Number of % of 
effect sizes studies studies 
Enhancement of academic skills 2 2 5.4 
After-school programmes 2 2 5.4 
Mentoring/monitoring 5 5 13.5 
Skills training for students i) i] 24.3 
Skills training for teachers 4 3 8.1 
School-wide strategies 6 6 16.2 
Violence reduction 3 3 8.1 
Counselling, mental health 3 3 8.1 
Other 4 4 10.8 
Total 38 37 100 


Enhancement of academic skills. We found two effect sizes targeting the enhancement of 
academic skills as a strategy in order to improve academic performance, increase motivation 
and promote more adaptive behaviour. Edmunds et al., (2012) tested an intervention to 
boost the academic progress of students in order to facilitate their future access to college 
while in the case of Cook et al., (2014), the intervention involved academic remediation plus 
social skills training. 


After-school programmes. Two effect sizes come from interventions that offered students 
after-school activities. While the intervention tested by Dynarski et al., (2003) was more 
focused on academic support and recreational activities, Hirsch, Hedges, Stawicki, & 
Mekinda, (2011) tested an after-school programme offering students paid apprenticeships. 


Mentoring/monitoring programmes. Five effect sizes reported the impact of interventions 
focused on mentoring/monitoring. These programmes involved structured and supportive 
relationships between a young person who presents academic, emotional or behavioural 
difficulties and a non-parental adult, their mentor. Mentoring entails a volunteer member of 
the community serving as a role model and providing support to a younger person over an 
extended period of time (e.g., Brett, 1993; Wyman et al., 2010). In other cases, adults served 
as tutors for the students, supervising their performance, providing advice or counselling, 
and assisting the students with academic tasks (Johnson, 1983; Peck, 2006; Reese et al., 
1981). Such tutors were normally schoolteachers or school counsellors. 


Social skills training for students. We found nine effect sizes representing the impact of 
social skills training. These programmes were based on social learning and cognitive 
behavioural theories (e.g., Burcham, 2002; Collier, 2002; Harding, 2011; Hostetler & 
Vondracek, 1995; Shetgiri, Kataoka, Lin, & Flores, 2011; Smith, 2004) and their goal is to 
enhance individuals’ socio-cognitive, socio-emotional, and behavioural skills in order to 
regulate maladaptive conducts. Social skills training programmes typically consist of a 
curriculum with focused training modules. Some more specific programmes target 
communication skills (e.g., Obsuth et al., 2016) or approaches to reducing stress (e.g., 
Barnes, Bauza, & Treiber, 2003). Group-based sessions and occasionally one-to-one sessions 
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(e.g., Russell, 2007) offer the opportunity to implement specific techniques (e.g., instruction, 
modelling, role-playing, feedback and reinforcement) in a “real-world environment”. 


Skills training for teachers. We found four independent interventions targeting teachers’ 
skills. The training provides preventive strategies and techniques that help to maintain 
classroom discipline, create a supportive educational environment, and enhance students’ 
positive behaviour. These involve training in facilitating mutual respect between teacher and 
student (e.g., Okonofua, Pauneskua, & Walton, 2016) as well as training to establish clear 
classroom rules (e.g., Hawkins, Doueck, & Lishner, 1988). Skills for teachers also involve 
strategies for working in an alliance with parents to promote students’ engagement to the 
school activities (e.g., Ialongo et al., 2001). 


School-wide interventions. Six effect sizes represent comprehensive interventions targeting 
systemic changes across the whole school. They involve pupils, teachers, parents, and 
sometimes also the community where the school is based. These programmes aim to create 
positive environments, with clear rules that promote good behaviour, learning and safety. 
School-wide interventions are capable of addressing the needs of schools as institutions as 
well as the particular needs of individual school children. We found six studies testing 
school-wide strategies: Bradshaw et al., (2012); Cornell et al., (2012); Lewis et al., (2013); 
Snyder et al., (2010); Sprague et al., (2016); Ward & Gersten, (2013). 


Violence reduction. We included three effect sizes measuring the impact of violence 
reduction programmes. Although these interventions could be classified as skills training, we 
have isolated them, because they are specifically targeted at increasing self-control and 
reducing violence (e.g., Feindler, Marriott, & Iwata, 1984; Mack, 2001). We also included 
anger management programmes encouraging peaceful responses to conflict (e.g., Farrell et 
al., 2010). 


Counselling and mental health interventions. We included three effect sizes primarily 
focused on the provision of counselling in schools (e.g., Berlanga, 2004; Tilghman, 1988) and 
on a more specialised provision from community mental health services (i.e., 
Panayiotopoulos & Kerfoot, 2004). 


Other interventions. Four effect sizes were classified in this general category. They encompass 
a community services programme (Allen, Philliber, & Herrling, 1997), a multicomponent 
programme (Arter, 2005), a career awareness intervention (Bragdon, 2010), and a programme 
focused on character-building education, promoting civic behaviour and national values 
(Crowder, 2001). 


4.2 RISK OF BIAS IN INCLUDED STUDIES 


The methodological quality of each publication included in the review was evaluated using 
the EPOC risk of bias tool (see section 12.3). The instrument evaluates the internal validity of 
reported results. Two coders (SV & AS) independently applied the EPOC tool to each study at 
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different locations. The following results represent the agreed rating of both coders.29 Below, 
we report the results for each of the eight criteria involved in EPOC. Figure 2 shows a 
summary of the overall result, and Table 9.2 (section 9.2 in appendix), offers a detailed 
evaluation of each study. 


Figure 2: Risk of bias in included studies based on EPOC risk of bias tool 
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Note. Each of the eight evaluated criteria have been assigned one of three possible alternatives: high risk= HR; low risk= LR 
and unclear risk= UR, as expressed in the first column underneath the graph. 


Adequate sequence generation 


Adequate sequence generation is intended to produce comparable groups in an experimental 
evaluation. Accurate methods used to generate the allocation are, for instance, the use of 
random number tables or a computer random number generator (Higgins & Green, 2011). 
Regardless of the selected methods, it should be clearly described in enough detail to allow an 
assessment of whether the sequence generation was adequate. 


As shown in Figure 2, only three studies (8%) presented a high risk of bias with respect to 
allocation sequence. A case in point: Allen, Philliber, Herrling, & Kuperminc, (1997) used 
three different methods of randomisation (i.e., picking names from a hat, coin toss and using 
an alphabetical list of names) as well as running randomisation at different levels in the same 
study (i.e., individuals and classrooms). In our opinion this mixture of methods could lead to 
a high risk of bias in the final results: in particular, the use of a list of names is known not to 
be appropriate for allocating cases. In the case of Barnes, Bauza, & Treiber, (2003:2), the 
author mentions that “each school was alternately assigned to either TM or health education 


29 At the beginning of the evaluation we selected two studies and coded them as a pilot assessment to ensure we 
understood the criteria consistently. This exercise helped to test the instrument and also to adjust some criteria. 
Following the complete coding of all studies, the coders met in late June 2016 to discuss and moderate their 
ratings. 
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control.” Since the study involves only two schools, allocation would be predictable and at a 
high risk of bias. 


Although we identify a low percentage of high-risk cases, it is important to note that a large 
proportion of studies (59%) were categorised as “unclear risk”. This is because many reports 
presented succinct descriptions of the randomisation process without detailing the methods 
of sequence generation (e.g., Arter, 2005; Brett, 1993; Feindler, Marriott, & Iwata, 1984; 
Reese, Murphy, & Filipezak, 1981). 32% of manuscripts were defined as presenting a low risk 
of bias (e.g., Bradshaw et al., 2012; Hirsch et al., 2011; Lewis et al., 2013). 


Allocation concealment 


Correct allocation concealment safeguards a rigorous implementation of the randomisation 
process by not allowing researchers or participants prior knowledge of the results of 
assignment (i.e., by using sealed envelopes or other procedure that prevent knowledge about 
the condition that the participant is going to be allocated). To achieve this aim it has been 
suggested that allocation should be centralised and executed at the beginning of the study 
(Schulz & Grimes, 2002). 


In the present meta-analysis, only 5% of the studies were evaluated as having a high risk of 
bias for allocation concealment. Some examples of high-risk studies are those where reports 
suggested that schoolteachers instead of researchers performed the random allocation (e.g., 
Crowder, 2001) or that the randomisation was performed after the participants’ screening 
(i.e., Mack, 2001). Even if we cannot be sure that the results of the screening biased the 
randomisation, this conduct could potentially interfere with the chances of participants being 
placed in the control or treatment group. 


Once again, we found a high percentage of studies (54%) reporting minimal details of 
allocation concealment. In concrete terms, 20 out of 37 included reports were classified as 
“unclear risk” of bias. The remaining 41% (i.e. 15 studies) were classified as having a low risk 
of bias. 


Baseline equivalence in the outcome measured 


A key element of a randomised controlled trial is that it ensures, in theory, that participants 
(and their associated outcomes and characteristics) are distributed by chance in the control 
and treatment groups. Pre-existing baseline differences between groups — particularly of 
outcomes — could suggest problems in randomisation, hence it is a key focus when assessing 
risk of bias (Shadish et al., 2002). It is important to mention that we did not focus on 
whether there were statistically significant differences between groups, as that is a function of 
sample size; instead we reviewed means and distributions. 


We found seven studies (19%) whose description of the baseline equivalence suggested a 
high-risk. This was found, for instance, in cases where the control group presented higher 
levels of exclusion than the treatment group and there was not clear mention of adjustment 
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(e.g., Panayiotopoulos & Kerfoot, 2004). Another example concerns the imbalance of one 
specific type of exclusion: specifically, expulsion and out-of-school exclusion were equivalent 
in the treatment and the control groups, but display substantial imbalance in the case of in- 
school exclusion (e.g., Berlanga, 2004). 


Again, it is important to mention that nearly 30% of assessed studies reported limited 
information, and for that reason they were assigned an unclear risk of bias. The remaining 
49% of our included randomised controlled trials were evaluated as low risk of bias. 


Baseline equivalence in other participants’ characteristics 


Regarding this criterion, the instrument considers the balance in demographics or any other 
behavioural outcomes, which again should be equivalent in the treatment and the control 
group if randomisation has been successful.3° We found 11 studies (30%) displaying non- 
equivalent results between treatment and control groups. Studies reporting imbalance in 
gender (e.g., Berlanga, 2004), ethnicity (e.g., Hostetler & Fisher, 1997), initial levels of 
problem behaviour, self-control or size of the school (e.g., Harding, 2011) were all classified 
as a potentially having a high risk of bias. 


A further 22% (eight studies) did not report enough data for judgement (i.e., unclear risk) 
and around half (49%) of the studies were assessed as presenting a low risk of bias. 


Addressing incomplete outcome data 


The fact that our included studies pursued more than one measure across time makes it likely 
that attrition or other forms of missingnes affected sample sizes. 


We assess a high risk of bias when i) substantial attrition was present in the study and the 
researchers did not mention a strategy to deal with that issue (e.g., Peck, 2006); ii) when they 
used list-wise deletion (e.g., Russell, 2007), and iii) in those cases where the attrition affected 
the treatment or control group in an unequal proportion of missing cases across arms (e.g. 
Hostetler & Fisher, 1997). Using these criteria, we judged that eight studies (22%) presented 
a high risk of bias. 


As in previous cases, a high number of studies (30%) did not report enough data to be 
judged. Seventeen studies (46%) were evaluated as presenting a low risk of bias. Low risk 
cases involved i) those studies reporting zero attrition; ii) studies where attrition was 
represented by a low percentage of cases; iii) when missingnes was equivalent in the 
treatment and the control group, and iv) when the researcher reported attrition, analysed it 
and used methods to deal with attrition (e.g., multiple imputation, full information 
maximum likelihood or intention to treat rather than assessing the effect of treatment on the 
treated). 


30 How many variables would we expect to be out of balance by chance in an RCT? This figure is usually obtained 
by the following calculation (k*0.05) where k is the number of variables, so 20*0.05 = 1. 
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Blinding of outcome assessment 


This criterion covers bias arising from the fact that those collecting outcome data are aware 
of the condition assigned to each participant (e.g. individuals, classrooms, schools). In our 
evaluation, 35% or 13 of the coded studies were assessed as having a high risk of bias. We 
assigned a high risk of bias when those in charge of delivering the intervention were also 
collecting data (e.g., Harding, 2011) or when teachers rated the students’ behaviour while 
being aware of the allocated condition (e.g., Tilghman, 1988). Although the outcome was in 
the majority of studies, based on official records, we did not assume this meant blind 
assessment. In most cases, teachers or school staff who imposed exclusion were not 
necessarily blind to the allocated condition of participants. In any case, it must be said that in 
school-based experiments blinding is likely infeasible. Most of the studies require at least a 
minimal participation of school staff and allocated condition is mostly evident for 
participants (Hutchison & Styles, 2010). 


As in previous cases, the number of reports lacking the data to evaluate blindness of the 
outcome assessment (i.e., unclear risk) was high (54%) and only four studies (11%) were 
assessed as low risk bias. 


Protection against contamination 


Protection against contamination refers to the measures taken to avoid a spill-over of the 
treatment into the control group. Specifically, contamination is the risk that the control 
group might accidentally receive the intervention, but also the risk that the two groups could 
influence each other’s outcomes. If the groups cannot remain isolated during the 
experimental study, contamination would threaten the validity of the results. In our meta- 
analysis 76% of the studies presented a high risk of contamination. The main explanation for 
high risk of bias was the fact that schools in the trials host both the treatment and the control 
participants (e.g., Cook et al., 2014; Smith, 2004). 


One single study (3%) did not report enough data for evaluation and it was categorised as 
unclear risk of bias. Eight studies (22%) were classified as low risk of bias. Normally low risk 
studies were cluster-randomised experiments where control and treatment participants were 
in different schools (e.g., Bradshaw et al., 2012; Snyder et al., 2010). 


Selective outcome reporting 


Selective outcome reporting occurs when there is a difference between the proposed 
outcomes for evaluation and those finally reported. In our evaluation, four studies (11%) 
presented a high risk of selective outcome reporting (e.g., Panayiotopoulos & Kerfoot, 2004; 
Russell, 2007). In the case of thesis or trials without published protocols, the assessment was 
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only based on discrepancies between outcomes proposed and reported in those documents?'. 
Thirty-two studies (86%) displayed a low risk of bias and only 3% did not report enough data 
for judgement. 


Summary 


Overall, our assessment demonstrates that a high number of the included studies lacked 
enough information to use the EPOC tool to judge risk of bias in all areas. Specifically, 22 
studies were not clear on how they allocated units to treatment and control groups. Similarly, 
blinding to allocation was another area infrequently reported with enough detail to allow 
assessment (i.e., in the case of 21 studies). These points aside, contamination or spill-over is 
probably the main threat to the validity of results among the RCTs included in our review. 


Arguably the seven methodological strongest studies were Bradshaw et al., (2012), Cook et 
al., (2014), Hirsch et al., (2011), Johnson, (1983), Lewis et al., (2013), Obsuth et al., (2016) 
and Wyman et al., (2010). All of them presented low risk in the randomisation process and 
most of them are clustered studies. In particular, these studies achieved low risk of bias in six 
or more of the eight EPOC criteria. 


4.3 SYNTHESIS OF RESULTS 


4.3.1 Primary outcome: overall impact of school-based intervention 


The present analysis incorporates 38 effect sizes across 37 studies producing enough 
statistical information for meta-analysis. These studies represent a total sample of 31,273 
students (Mage=12.5; SD=2.85) partaking in completed trials as treatment, control or placebo 
groups. 


On average, school exclusion was significantly reduced in the experimental group compared 
with the control group, post-treatment (i.e., six months on average). Under a random effects 
model, the standardised mean reduction was SMD=.30; (95% CI .20 to .41; p<.001). Figure 3 
shows that results are positive and statistically significant, meaning that those participating 
in school-based interventions were less likely to be excluded than those in the control group. 
Results exhibit significant heterogeneity (Q=301.3; df= 37; p<.001; [?= 87.7; T?=.078) which 
was expected in this meta-analysis bearing in mind that we include different school-based 
programmes, administering different “doses of treatment” with participants from different 
locations and in different school grades. 


31 Many of our included studies were non-published PhD thesis (i.e., 18 out of 37). As a result, it was impossible to 
find pre-registration publications for these studies. We observed that most of the theses reported small or null 
effects (i.e., Arter, 2005; Bragdon, 2010; Brett, 1993; Burcham, 2002; Crowder, 2001; Harding, 2011; Peck, 
2006). Since null and small effects were reported, we assumed low risk of bias regarding “selective outcome 
reporting”. 
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Figure 3: Forest plot of the effect sizes for the impact of school-based programmes on school 
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As suggested by Lipsey and Wilson (2001, p. 153) the U3 statistic for a SMD= .30 is 62%. 
This would indicate that 62% of the treatment group is above the median of the control 


group. Stated differently, if we assume a 50/50 success rate for both groups, the treatment 
group sees 62% success, versus 38% in the control group. 


Long-term effects 


When we isolated studies measuring impact at follow-up (i.e., 12 or more months after 


finishing the intervention), benefits of the interventions were less clear. In fact, Figure 4 
demonstrate that the effect was reduced by half (SMD=.15; 95%CI -.06 to .35) and it was 
shown to be non-statistically significant. 
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Figure 4: Forest plot of the effect sizes for the impact of school-based programmes on school 
exclusion: long-term effects 
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To increase precision in our results, we ran a meta-analysis with a subset of studies reporting 
both post-treatment and follow-up measures. Only seven studies reported short and long- 
term effect measures. As observed, the overall effect size at post-treatment (Figure 5) and 
follow-up (Figure 6) are lower than the ones initially reported but they follow the same 
direction. 


Figure 5: Forest plot of the effect sizes for the impact of school-based programmes on school 
exclusion: post-treatment (seven studies only) 
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Figure 6: Forest plot of the effect sizes for the impact of school-based programmes on school 
exclusion: follow-up (seven studies only) 
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Stddiff Standard Lower Upper 
in means error Variance — limit limit Z-Value p-Value 
Dinarsky etal 2003 Suspension -0.122 0.118 0.014 -0.353 6.109 -1.035 0.301 
Lewis etal 2013 Suspension 0,007 0.063 0.004 -0.117 0.131 0.111 0.912 
Hostetler etal 1997 Suspension 0.022 0.100 0.010 -0.174 0.218 0.220 0.826 
Farrell etal 2001 Combined 0.119 0.201 0.040 -0.275 0.513 0.592 0.554 
Ward 2013 Suspension 0,144 0.078 0.006 -0.010 0.297 1,838 0.066 
Burcham 2002 Combined 0.161 0.216 0.047 -0.262 0.584 0.747 0.455 
Snyder 2010 Suspension 0,605 0.305 0.093 0,007 1.203 1,984 0.047 
0.054 0.049 0.002 -0.043 0.151 1.097 0.273 


~4.00 2.00 0.00 2.00 4.00 


Favours Control Favours Treatment 


At post-treatment, under a random effect model, the standardised mean reduction was 
SMD=.21 (95%CI .11 to .30). However, when we ran the meta-analysis including only the 
subset of seven studies, the average time involved under “post treatment” was 12 months. It 
implies that although the overall effect is slightly lower than the general measure reported in 
Figure 3, the impact lasts longer (i.e., 12 instead of 6 months on average). In the case of the 
effect at follow-up, the subset of studies produced and overall impact that was almost null 
(SMD=.054; 95%CI -.04 to .15) and non-significant. As shown in Table 11, heterogeneity was 
highly reduced when the analysis with the subset of studies was carried out. 


Table 11: Summary of overall effect at post-treatment and follow-up (seven studies only) 


Impact SMD 95% Cl n k Measure of Heterogeneity 
Post- 21 (.11; .30) 7 7 = Q=6.54; df=6; p>.05; /2=8.2; 12=.001 
treatment 


Follow-up .054 = (-.04; .15) 7 7 Q=7.80; df=6; p>.05; 1?=23; 12=.004 


4.3.2 Moderator analysis by type of exclusion 


As proposed, we ran an analysis for the different outcome measures: in-school exclusion, out- 
of-school exclusion, expulsion, and general exclusion. Results of the four independent meta- 
analyses are summarised below in Table 12. 
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Table 12: Results of four independent meta-analyses by type of disciplinary exclusion 


Type of SMD 95% Cl n k Measure of Heterogeneity 
Exclusion 

In-school 39 (.11; .58) 6 6 Q=11.62; a5; p<.05; 12=57; 7=.045 
Out-of-school 02 (-.16; .19) g g Q=22.72; df=8; p<.05; 12=65; 17=.041 
Expulsion 53 (.07; .98) 4 4 Q=13.66; df= 3p<.05; |2=78; 12=.14 
General measure 32 (.21 43) 27 28 Q=171.4; df=27 p<.001; I2=84; 12=.056 


Note. n= number of studies; k=number of effect sizes. 


In-school exclusion. Only six trials were concerned with the effect of school-based 
intervention on reducing in-school exclusion. Under a random model, the effect of the 
intervention remained positive and statistically significant. The final effect described in 
Figure 7 was SMD=.35 (95% CI .11 to .58; p<.005). Once again heterogeneity was tested, 
finding significant but smaller variability across studies (Q=11.62; df=5; p<.05; I?=57; 
T?=.045). 


Figure 7: Forest plot of the effect sizes for the impact of school-based programmes on in- 
school exclusion 


Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Std diff Standard Lower Upper 

in means error Variance limit limit Z-Value p-Value 
Burcham 2002 = In school 0.030 0.170 0.029 40.304 0.364 0.176 0.880 
Crowder 2001 = In school 0.110 0.192 0.037 0.206 0.486 0.574 0.566 
Sprague 2016 = In school 0.280 0.100 0.010 0.084 860.476 2.800 0.005 
Berlanga 2004 = In school 0.386 0.179 0.032 0.0385 0.737 2.158 0.031 
Farrell 2001 In school 0.887 0.342 0.117 0.216 1.558 2.592 0.010 + 
Smith 2004 In school 1.050 0.337 0114 0389 1711 3411 0.002 +- 

0.345 0.120 0.014 0.110 0.580 2.877 0.004 ¢ 
4.00 2.00 0.00 2.00 4.00 


Favours Control Favours Treatment 
Out-of-school exclusion. Correspondingly, nine studies reported data for the impact of 


interventions on out-of-school exclusion. Figure 8 shows an effect close to zero, non- 
statistically significant of SMD=.02 (95% CI -.16 to .19; p>.05). 
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Figure 8: Forest plot of the effect sizes for the impact of school-based programmes on out- 
of-school 


Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Std diff Standard Lower Upper 
in means error Variance — limit limit Z-Value p-Value 
Sprague 2016 Out of school 0270 0.100 0.010 0466 074 -2.700 ‘007 
Mack 2001 Out of school 0232 0318 0.101 0.855 0.381 0730 0.465 
Bracsmaw 2012 Out of school 0174 0.051 2.004 033 +0054 -2aMT 2.004 
Farrell 2001 Out of school 40.058 0.188 0.036 40.428 0312 0.308 758 
‘Crowder 2001 Out of school 023 a1g2 037 UE 0.405 0.152 ars 
Burenam 2002 «= ut of school 0.067 170 ozs ET 0.401 0.383 ees 
Berianga 2004 Qutofschoal 0.180 ars 032 471 0531 1.006 o314 
Cook 2014 Out of school a3s4 0276 076 4187 aes 1.281 0.200 
‘Sem 2004 Out of school 1.040 0335 at2 Oss = 1.696 3.108 o.o02 -i- 
0018 aos2 0.008 0165 as 0.191 0.845 
4.00 2.00 0.00 2.00 4.00 
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Expulsion. The impact of school-based interventions on expulsion was significantly higher 
than any other impact described so far. Figure 9 shows that expulsion was reduced by 
SMD=.53 (95% CI .07 to .98; p<.05) with significant heterogeneity (Q=13.66; df=3; p<.05; 
[?=78; T?=.14) but based on only four reports presenting data for analysis. Therefore, these 
results must be evaluated cautiously. 


Figure 9: Forest plot of the effect sizes for the impact of school-based programmes on school 
expulsion 


Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Std diff Standard Lower Upper 

in means error Variance limit limit Z-Value p-Value 
Harding 2011 Expulsion 0.081 0.799 0.638 -1.484 1.646 0.101 0.919 
Sprague 2016 Expulsion 0.230 0.100 0.010 0.034 0.426 2.3200 0.021 
Berlangs 2004 Expulsion 0.385 0.179 0.032 0.034 0.736 2.152 0.031 
Feindler 1584 Expulsion 1.203 0.245 0.060 0.723 1.683 4.911 0.000 iii 

0.527 0.233 0.054 0.070 860.985 2.261 0.024 & 
4.00 -2.00 0.00 2.00 4.00 


Favours Control Favours Treatment 


General Suspension. Finally, a number of studies presented data on suspension as a broad 
and general measure. These studies did not describe operational definitions about the type of 
disciplinary suspensions involved in the outcome. We have defined this category as General 
Suspension. In the aim of transparency, we report these results separately, although this 
measure could involve any of the previous outcomes reported above and, therefore, be a 
subset of the overall effect size reported at the beginning of this section. 
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Figure 10 shows 27 studies reporting 28 independent effect sizes concerned with the impact 
of targeted interventions on general suspension. The effect of school-based interventions was 
positive SMD= .32 (95% CI .21 to .43; p<.001), it was statistically significant and similar to 
the overall effect size reported in Figure 3 above. In addition, heterogeneity remained 
substantial (Q=171.45; df=27 p<.001; [?=84; T?=.056). 


Figure 10: Forest plot of the effect sizes for the impact of school-based programmes on 
general school suspension 


Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Std diff Standard Lower Upper 

inmeans error Variance limit limit Z-Value p-Value 
‘Snetguirl 2011 Suapension or Expulsion 0.388 0251 00s -0890 0084 = -1.556 0113 
Oneuin 2016 Suspension 2 oat oom 0639 «0085 = -2.560 0.010 
Amer 2005 Suspension 0.000 0320 o12 8-627 = O62 0.000 1.000 
Peck 2006 Suspension oot o.172 ooo a7) 86M 0.065 oss 
Hirson 2011 Suspension 0.023 oort 000g 6-016 ©0162 0325 ons 
Bragoon 2010 ‘Suspension 0.058 0.176 0031 0287 = 0.403 0329 or 
Bret 1983 Suspension 090 0265 ooro 0423 68 0340 OTs 
Ward 2013 Suspension 0.100 0087 oo 1068 0270 1.158 0245 
Harding 2011 Suspension 0101 0.388 ois 1680 90.881 0252 0801 
Russel 2007 Combined 0154 0.400 oreo 0631 890.88 0364 a701 
islongo 2001 SuspensionOC o1e8 ons oo 6-014 = 0.353 1.805 oort 
Hostetler 1937 Suspension 0.196 0.84 oo acsz 0.360 203 0019 
Diareky 2003 Suspension 0.198 0.168 omg -1133 860530 1.172 oat 
Hawking 1985 ‘Suspension 0232 0.168 oom 8 -1099 «= OSe2 1.375 0.169 
islongo 20010 SuspenskonF SM 0209 0.176 0031 -0056 ©0635 1.641 0.101 
Lewis 2013 Suepension 0330 0.100 oni 860134 = 0526 3300 0.001 ri 
Allen, 1937 Suspension OMS our7 000 8 6a19f o4g5 4.428 0.000 |_| 
Panaylotopoulos2004 Suspension oats 0.126 0016 8 a170 60666 3.305 0.001 i 
Eomunds 2012 ‘Suspension 0440 0.100 ooi0 860244 = OB36 4.400 0.000 |_| 
‘Omonofua 2016 Suspension o47a on ooo 60451) = 805 34.680 0.000 fe] 
Reese 1991 Suepension osa7 o212 ous ai20 0853 2526 0012 i 
Comell 2012 Suspension 0583 0.168 ous 0253 O83 3.464 0.001 SF 
‘Smyder 2010 Suspension 0610 0316 0100-0010 1.230 129 0.054 
Wyman 2010 ‘Suspension 0683 0.408 O16 0141 1.458 1.615 0.106 
Tiighenan 1958 Suspenskn aati 0253 oo ass 1307 3.206 0.001 -- 
Bames 2003 Suspension asi3 O26 oot 8 86a470 (1.386 4083 000 -—i- 
Coller 2002 Suspension 1.083 o1g2 007) 0 a7zt 1.476 5713 0,000 i 
Jonneon 1983 Suspension 1.089 0.182 oor 6 ara? 1.476 5713 0.000 - 

0316 0.056 0003 ame a4a7 5621 0.000 + 


4.0 -2.00 0.00 2.00 4.00 


Favours Control Favours Treatment 


4.3.3 Secondary outcome: overall impact of school-based intervention on 
internalising behaviours 


As stated in our protocol, for any identified study reporting data on school exclusion, we also 
coded a secondary outcome referring to internalising and externalising behaviours. Data on 
internalising behaviour was not very prolific in our set of included studies. Only five trials 
presented statistical results but sometimes the data was insufficient for effect size 
calculations. For that reason, we are not running meta-analysis on internalising behaviours, 
but we summarise the results in Table 13. 
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Table 13: Impact of school based interventions on internalising behaviours 


Study Sample Measure Statistical measures 
Bradshaw N=12,334 Emotion regulation Students in SWPBIS school fared better in comparison with 
2012 control schools. (y=.05; t=2.38; p<.05) 
Harding 2011 N=43 Emotional Symptoms Negative effect, not statistically significant. 
(SDQ) SMD=-.30 (95%Cl -.3 to .91); p>.05 
Russell 2007 N=61 Internalising problems On page 20, Table 4, the author presents means, standard 


deviations and sample size for treatment and control group. 
The author asserts that intervention reduced antisocial 
behaviour. Based on our calculations, that reduction is not 
significant (SMD=.32; 95%Cl -.14 to .79; p>.05) 


Tilgham 1988 N=100 Anxiety The definition of the measure suggests that anxiety is part of a 
composite measure. Impact of treatment on anxiety is unclear 
(p.49). 

Wyman N=226 Assertive vs. withdrawn Measure is described as a measure of anxiety (e.g., “Nervous, 


frightened”). The programme has a positive effect on the 
internalizing behaviour ES=.37 (.03 to .71). 


4.3.4 Secondary outcomes: overall impact of school-based intervention on 
externalising behaviours 


We found a diverse range of measures referring to externalising behaviours, such as 
substance misuse, violence, aggression, and problematic behaviour in school. To pool 
together comparable measures, we ran a meta-analysis only on behaviours that could be 
categorised as antisocial such as aggression, physical fights, delinquency, bullying and 
conduct disorder?2. 


Fourteen studies reported complete data for a composite measure of antisocial behaviour 
(See Figure 11). The fourteen studies provided 15 independent effect sizes. Unusually, 
Wyman et al. (2010) reported a measure of behaviour control (e.g., children accepting 
imposed limits). We reversed the results as a proxy of antisocial behaviour and included this 
information in our calculations. The same procedure was followed with Feindler et al. (1984) 
who reported a measure of increase in self-control. Once again, we reversed the effects size 
as a proxy of antisocial behaviour. 


As reported in Figure 11, the overall impact of school-based interventions on antisocial 
behaviour, under a random effects model, was not statistically different from zero SMD= - 
.005 (95% CI -.097 to .09; p >.05) indicating an overall null effect of these programmes at 
reducing antisocial behaviour. 


32 Substance misuse was not included in the calculation. 
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It must be highlighted that some of the included studies in this overall measure reported 
negative effect sizes, meaning that in some specific cases the intervention had iatrogenic 
impact (Hawkins et al., 1988; Hostetler & Fisher, 1997; Obsuth et al., 2016). For instance, 
Hawkins et al., (1988) reported the impact of an intervention focused on interactive teaching 
and co-operative learning targeting low achievers in mainstream schools. The treatment 
group showed a reduction in the number of exclusions but an increase in the mean value for 
serious crime. The study reported by Hostetler & Fisher, (1997) described a similar case. For 
its part, Obsuth et al., (2016) reported the results of a clustered randomized controlled trial 
testing the impact of “Engage in Education-London”. The intervention targeted high risk 
students and it was aimed at improving communication and broader social skills. Results 
suggest that the iatrogenic effects were observable not only for exclusion but also for the case 
of antisocial behavior. 


It is important to mention that Feindler et al., 1984; as well as Wyman et al., (2010) were 
reverted and used as proxies of antisocial behaviour. For this reason, they should not be 
understood as interventions which report iatrogenic results. 


Figure 11: Forest plot of the effect sizes for the impact of school-based programmes on 
antisocial behaviour 


Stuiyname = Duicome:_ _Sintinties for-enck stitty SET iy emans are BI 
Std diff Standard Lower Upper 
in means error Variance limit limit Z-Value p-Value 
Hawkins etal 1988 Serious crime -0.231 0.176 0.031 -0.576 0.114 -1.310 0.190 
Wyman etal 2010 Behaviour control ~-0.224 0.133 0018 -0486 0.037 -1.681 0.093 
Feindler etal 1984 — Self-control -0.165 0.239 0.057 -0.633 0.303 -0.691 0.489 
Hostetler etal 1997 Negative Behaviour -0.161 0.084 0.007 -0325 0.003 -1.924 0.054 
Obsuth etal 2016 = Antisocial -0.160 0.055 0.003 -0.267 -0.053 -2.921 0.003 
Sprague etal 2016 Antisocial -0.012 0.071 0.005 -0.151 0.127 -0.170 0.865 
Burcham 2002 Problem Behaviour 0.014 0.056 0.003 -0.095 0.123 0.251 0.801 
Hirsch etal 2011 Physical Fight 0.022 0.071 0.005 -0.117 0.161 0.311 0.756 
Dynarski etal 2003 Negative Behaviour 0.061 0.354 0.126 -0.634 0.755 0.171 0.864 
Shetguiri etal 2011 Delinquency 0.145 0.173 0.030 -0.194 0.484 0.837 0.403 
lalongo etal 2001b Conduct disorderFS 0.201 0.276 0.076 -0.340 0.743 0.729 0.466 
Ward etal 2013 Bullying 0.329 0.256 0.066 -0.173 0.832 1.284 0.199 
Farrell etal 2001 Violence 0.435 0.176 0.031 0.089 0780 2467 0.014 
Collier 2002 Oppositional 0.478 0.302 0.091 -0.113 1.069 1.585 0.113 
lalongo etal 2001 Conduct disorderCC 0.478 0.302 0.091 -0.114 1.070 1.582 0.114 
-0.005 0.047 0.002 -0.097 0.086 -0.114 0.909 
-8.00 -4.00 0.00 4.00 8.00 
Control Treatment 


4.4 SUB-GROUP ANALYSIS 


A number of potential effect modifiers that could help to explain the expected heterogeneity 
in our results were initially identified in the published protocol. Based on previous research 
we pre-defined moderators that covered four aspects, namely: i) participants’ demographic 
characteristics; ii) behavioural problems; iii) the theoretical basis of the interventions, and iv) 
the quality of the intervention. In this section we present sub-group analysis results. The 
calculations have been run under a random effects model assuming a separate variance 
component. Forest plots with further details are reported in Section 11 (appendix). 
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4.4.1 Effects moderated by participants’ demographic characteristics 


Of the 38 effects reported in this meta-analysis, 11 were tested in schools whose population 
was predominantly male (i.e., more than 60% of students were male), and 19 effects were 
tested in schools presenting a mixed population (i.e., neither gender exceeded 60 per cent). 


Post-intervention effects were different for both groups. For studies targeting predominantly 
male schools, the standardised mean difference was SMD=.41; (95% CI .10 to .72; p<.05). In 
studies targeting mixed-gender schools, the impact was lower (SMD=.17; 95% CI .02 to .32: 
p<.05). Differences between groups were not statistically significant (Q=1.84; df=1; p>.05). 


In terms of age, the best proxy variable was school grade. This information was reported in 
34 studies and 35 independent effects. To test the hypothesis that effect sizes vary by age, we 
ran sub-group analysis for 12 studies involving elementary school students (SMD=.27; 95%CI 
.09 to .45; p<.05), 16 targeting middle schools (SMD=.23; 95% CI .04 to .41; p<.05), and 
eight targeting high schools (SMD=.45; 95%CI .18 to .72; p=.001). The effect was statistically 
significant in each sub-group and larger in high school populations. However, the between 
effect difference was not statistically significant (Q=1.81; df=2; p=.41) meaning that there is 
no evidence that the effect of school-based interventions differs by age (i.e., grade at school) 
in the present meta-analysis. 


Ethnicity was reported as a continuous variable (i.e., percentage of each ethnic group). For 
that reason, the role of ethnicity in explaining the effect of school-based programmes will be 
explored in a meta-regression analysis. 


4.4.2 Effects moderated by participants’ behavioural problems 


In our review, only a limited number of studies presented data on behaviour (.e., 
internalising or externalising behaviours). In addition, the overall impact of school-based 
interventions on antisocial behaviour was not statistically different from zero. 


Pupils involved in included studies shared a similar high-risk condition. They were registered 
in schools with a high percentage of ethnic minorities and more than 60% of the students in 
those schools received free school meals, which is an indicator of disadvantaged socio- 
economic backgrounds. In that sense, participants do not display high variability. 


4.4.3 Effect of different school-based programmes on school exclusion 


One of the aims of the present meta-analysis was to compare the effect of different 
interventions on the reduction of school exclusion. Table 14 presents the standardised mean 
differences, confidence intervals and p-values as well as measures of heterogeneity for each 
of the nine types of programmes included in the review. The typology is the same as 
described in section 4.1.7. 
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Table 14: Effect size by type of school-based intervention 


Type of SMD 95% Cl _—p-value k Measure of Heterogeneity 
Intervention 

Enhancement of 43 (.25; .61) — p<.001 2 Q=.09; d1; p>.05; /2=0; r2=0 
academic skills 

After-school 05 (-.08;.17)  p>.05 2 Q=.91; df1; p>.05; 12=0; r=0 
programme 

Mentoring/monitoring A7 (.02; .93) p<.05 5 Q=20.1; df4; p<.001; 12=80; 12=.21 
Skills training for 31 (-.05; .67) — p>.05 9 Q=60.4; df=8; p<.001; /2=86; 12=.23 
students 

Skills training for 131 (.11; .52) p<.05 4 Q=13.6; df3; p<.05; 12=78; 12=.03 
teachers 

School-wide 20 (-.03;.43)  p>.05 6 Q=34.8; df5; p<.001; 12=86; 12=.06 
strategies 

Violence reduction 48 (-.33; 1.3) p>.05 3 Q=13.3; df2; p<.001; 12=85; 12=.44 


Counselling, mental 46 (.23; .68) — p<.001 3 Q=2.65; df2; p>.05; 12=25; 1=.01 
health 


Other 21 (.03; .39) p<.05 4 Q=4.11; a3; p>.05; 12=27; 12=.01 


First of all, as observed in Table 14, most of the interventions are represented by a restricted 
number of effect sizes, and for this reason, these results should be interpreted cautiously. 


Secondly, the standardised mean differences of only five types of programmes present 
positive (small to moderate effect sizes) and statistically significant results in favour of the 
reduction of school exclusion. Those programmes are: i) Enhancement of academic skills, ii) 
Mentoring/monitoring, iii) Skills training for teachers, iv) Counselling/mental health 
services, and v) Other programmes. Since “other programmes” involve a mixture of different 
interventions we believe they cannot be interpreted in the same way as the remaining four 
types. Similarly, when it comes to the number of studies included in each sub-group, it seems 
that the most stable results are Mentoring/monitoring and Skills training for teachers since 
they are based on a larger number of studies. 


Thirdly, to test the hypothesis that differences were significant among the compared sub- 
types, we ran further analysis. The comparison demonstrates that differences are statistically 
significant (Q=18.4; df= 8; p<. 05), meaning that variation in effect sizes can be explained by 
the type of intervention implemented. 


7 The Campbell Collaboration | www.campbellcollaboration.org 


4.4.4 Theoretical bases 


Reported information on the theoretical bases of the interventions was not very 
comprehensive. Selected studies described components of interventions more frequently 
than reporting the theory or set of theories framing the praxis. Based on the provided details, 
20 interventions were clearly based on a cognitive behavioural frame, while another five were 
concerned with ecological ideas, targeting a change of the school system as a whole. The 
remaining effects refer to different theories such as emotional intelligence (i.e., Smith, 2004), 
empathy (i.e., Okonofua et al., 2016), civic values (i.e., Crowder, 2001) or developmental 
theories (i.e., Allen et al., 1997). A number of studies did not provide enough information to 
make a judgement. We tried to run sub-group analysis on the theoretical bases of the 
interventions but it demonstrated low power, given that the number of effects was, in some 
cases, as low as one per sub-group. In that scenario, we decided that a measure of the level of 
targeted change could inform more consistent data on the theory framing the interventions. 
We then divided the studies into those targeting a systemic change versus those targeting a 
change in pupils. 


The standardised mean difference for the 10 evaluations targeting a change at school level 
provided a significant reduction with a value of SMD=.25 (95%CI .04 to .45), whereas the 28 
evaluations targeting a change at the pupil’s level reported a significant reduction with a 
value of SMD=.33; (95%CI .19 to .48). Both independent effects are statistically significant; 
however, the between-group comparison reported non-significant differences (Q=.48; df=1; 
p>.05) meaning that there is no evidence that the effect differs by level of targeted change. 


4.4.5 Moderator analysis: quality of implementation 


Previous research demonstrates that well-implemented programmes — those including 
training, monitoring and supervision — display larger and more consistent effect sizes (e.g., 
Durlak et al., 2011; Gottfredson & Wilson, 2003; Losel & Beelmann, 2006). We tested this 
hypothesis based on two variables, namely, “training before implementation” and 
“monitoring during intervention”. 


Twenty-five studies (reporting 26 independent effect sizes) clearly stated the presence of 
training hours before the intervention was delivered (i.e., training hours for those delivering 
the intervention). In the remaining 12 studies, authors did not mention any kind of training. 
We ran sub-group analysis to test the hypothesis that those that reported training could 
produce a significantly different effect. In the end, those reporting prior training yielded a 
result equal to SMD=.29 (95% CI .16 to .43; p<.001) whereas those that did not report 
training produced an effect equal to SMD=.34 (95% CI .15 to .53; p<.001). Both effects were 
positive and significant, however the test of the difference between the two sub-groups of 
studies yielded a result of Q=.16 with df=1 and p>.05, meaning that there is not enough 
evidence that effects differ by presence/absence of prior training. 
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Fifteen studies that reported monitoring the implementation of the programme during the 
trial yielded a result equal to SMD=.20 (95%CTI .05 to .35; p<.05). In parallel, the 23 studies 
that did not report monitoring produced an SMD=.37 (95%CI .25 to .50; p<.o01). Both 
results were positive and statistically significant. The test for differences between sub-groups 
showed non-significant differences (Q=2.89; df=1; p>.05). 


4.4.6 Post-hoc moderators 


Based on descriptive analysis of data, and also based on previous research findings 
(Beelmann & Losel, 2006; Farrington, Ttofi, & Lésel, 2016), we selected three post-hoc 
moderators, namely: i) reasons for conducting the research; ii) evaluator role, and iii) risk of 
quality bias. Their results are described below: 


Reasons for conducting the research. We coded data related to the reasons for 
conducting research. Two categories are explored: i) demonstration study, referring to those 
studies testing the impact of an intervention under highly controlled optimal conditions and 
ii) routine evaluation, focusing on testing established programmes under circumstances that 
approach real-life conditions (Singal, Higgins, & Waljee, 2014). 


Sub-group analysis produced different effects for the 16 studies conducted for demonstration 
(SMD=.43; 95% CI .26 t0.59; p<.001), and the 18 studies carried out for routine evaluation 
(SMD= .13; (95% CI .00 to .25; p<.05). Demonstration studies reported the highest effect. 
The between-studies comparison results were significant (Q=8.15; df=1; p<.05), meaning 
that the effect varies depending on the reasons for conducting the research. 


Evaluator role. We coded data identifying the role of the evaluator. Independent 
evaluators were those not taking part in the design or implementation of the evaluated 
programme. Dependent evaluators were those also contributing to the design and/or the 
implementation of the programme. 


The 18 trials carried out by independent evaluators produced an SMD= .13 (95% CI .00 to 
.253; p<.05). Unsurprisingly, based on what is known about developer-led trials, the 16 RCTs 
conducted by dependent evaluators (i.e., those who also developed and/or designed the 
intervention) yielded results where SMD=.47 (95% CI .32 to .62; p<.001). The between- 
groups comparison demonstrated that effect sizes are lower for studies conducted by 
independent evaluators. These differences were statistically significant (Q=12.36; df=1; 
p<.001). 


Risk of quality bias. As reported in section 4.2, we ran EPOC tool for assessing risk of 
quality bias. Results show that studies vary in the amount of information available for 
judging their level of risk as well as their risk of bias. We speculated that the quality of studies 
could explain some of the heterogeneity present in our results. In order to use the results 
reported by EPOC, we transformed the “low risk”, “unclear risk” and “high risk” into 
continuous 0, 2 and 3 values respectively. The sum of the results of the eight EPOC criteria 
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resulted in an index that, in our case, was used as a representation of the level of risk of each 
single study. We tested the role of RoB in meta-regression. 


4.5 META-REGRESSION 


In order to explore heterogeneity, we ran meta-regression using the moderators defined a 
priori (participants’ characteristics and intervention characteristics). Model I included 
participants’ characteristics only, to get a sense of the net value of these variables in the 
results. Students’ gender, grade at school or ethnicity (continuous variable), as reported in 
Table 15, did not explain heterogeneity in the study results. In model II, we introduced the 
intervention characteristics (i.e., interventions targeting a change at the individual level 
versus those targeting a change at the school level). Once again, none of the variables present 
significant results. This suggests that based on the present data, variability across effect sizes 
cannot be explained by the a priori defined modifiers. 


Table 15: Meta-regression results: a priori moderators 


Predictors Model | Model Il 

b SE 95%Cl b SE 95%Cl 
Intercept oT 33 -.28; 1.06 37 134 -.30;1.04 
Participants’ characteristics 
Gender 
Male predominant .09 38 -.64;.82 11 39 -.66; .87 
Mixed -.05 3 -.17:.67 -.06 38 -.81; .69 
Grade at school (High) 
Elementary -.14 21 -.52;.29 -.16 23 -.61; .29 
Middle -.22 20 -.60;.16 -.24 20 -.64; .16 
Mixture A7 A0 -.62;.96 04 A8 -.90; .98 
% of White -.00 .004 -.008; .006 -.00 .003 -.00; .00 


Intervention characteristics 
Individual versus school level 
change 11 a2 -.30; .52 


* p<.05 


We proceed with a meta-regression model involving post-hoc variables that could potentially 
explain the heterogeneity present in our results. Based on the descriptive results in section 
4.1.6, and bearing in mind some previous findings, we hypothesise that the characteristics of 
the research, the ends of the research (demonstration versus routine), the role of the evaluator, 
and the risk of quality bias could play a role in explaining the results (see Table 16). 
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Table 16: Meta-regression results: post-hoc moderators 


Post-hoc predictors Model 

b SE 95%Cl 
Intercept 32 17 -.01; 
Characteristics of the research 
The ends of the research .06 14 -.20; 33 
(routine vs. demonstration) 
Role of the evaluator -.36 14 -.63; -.09* 
(independent vs. dependent) 
Risk of quality bias .008 01 -.01; .03 

*p<.05 


The results were significant (p<.05) only for the variable role of the evaluator. The coefficient 
is negative with the category of reference “dependent”, meaning that the effect is lower when 
an independent team runs research. 


Figure 12 presents the value of R-squared or percentage of explained variance between- 
studies based on study-level characteristics. To compute the total variance (of all studies 
about the grand mean), we run the regression with no covariates (a). To compute the 
variance not explained by the model (of all studies about the regression line), we run the 
regression with covariates (b). Finally, the difference between these values gives us the 
variance explained by the model which is R2=.58, meaning that the model explains 58% of 
the between-studies variance. 


Figure 12: R-squared graphic. Proportion of the variance explained by the role of the 
evaluator 


Total variance in true effects (a) 
0.0870 


Explained (c) 0.0503 


Total (a) = “0.0870 


Not explained by model (b) Explained by model (c) 
0.0367 0.0503 


4.6 PUBLICATION BIAS ANALYSIS 


Publication bias in systematic reviews occurs when the included set of manuscripts fail to 
systematically represent the whole population of completed studies that should have been 
included. The whole population of studies can involve a range of results that must be present 
in a meta-analysis to make it valid. However, consistent evidence indicates that studies 
presenting large effects are more likely to be published than those presenting null or modest 
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effects (Rothstein, Sutton, & Borenstein, 2006). This means that publication bias can lead to 
an overestimation of the impact of a treatment when running meta-analysis. 


In the present study, much effort was spent in finding most complete collection of published 
and unpublished studies that test the impact of school-based intervention on exclusion rates. 
In fact, almost 50% (see Table 7) of our included studies have never been published in books 
or peer reviewed journals (i.e., PhD thesis, technical or governmental reports). 


As originally proposed, we use statistical procedures to quantify potential bias that could 
affect our analysis. First of all, we produce a funnel plot of standard errors by standardised 
differences in means, presented in Figure 13. In theory, symmetrical distributions of dots 
under the funnel represent a normal distribution of studies. As anticipated, our studies 
mostly fall under the funnel, and they are distributed around the main effect. However, since 
the evaluation of funnel plots can be subjective, we conducted additional statistical measures 
of publication bias, specifically, Duval and Tweedie's trim-and-fill analysis. 


Figure 13: Funnel plot of standard error by standard differences in means 
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Duval and Tweedie's trim-and-fill analysis compares the differences in effect sizes that could 
potentially be attributed to bias. The technique imputes effect sizes until the error 
distribution gets close to normality. In this way, the test offers the best estimate of the 
unbiased effect (Borenstein et al., 2009). Results of Duval and Tweedie's trim-and-fill 
analysis suggest that there were no differences in effect sizes attributable to bias. Under the 
random effect model, the values were SMD=.30; 95%CI .20 and .41. Based on the parameter 
of Duval and Tweedie’s trim-and-fill, it seems that no studies are missing. 


82 The Campbell Collaboration | www.campbellcollaboration.org 


4.7 SENSITIVITY ANALYSIS 


In the present meta-analysis, the effect of the treatment was calculated as the difference 
between post-treatment and baseline. We corrected the value of the variance by assuming a 
value of correlation equals to .75. In order to test the robustness of this assumption, we ran 
sensitivity analysis with a correlation equal to .50. Table 17, panel A shows that overall results 
remain stable when the correlation is smaller. 


Another decision was related to the presence of outliers. We found one study presenting an 
effect size more than three standard deviations from the mean effect size, which was defined 
as an outlier (Collier, 2002). We tested the impact of the outlier and also the impact of 
winsorization. The size of the effects, their direction and significance were not altered. See 
Table 17 below, panels B and C. 


Table 17: Sensitivity analysis 


Panel A: with outlier 
Covariate SMD 95%Cl SE V n k Measure of Heterogeneity 


0.50 0.31 (.20;.43) 0.058 0.003 36 37 Q=259.0; dF36; p<.001; /2=86; r2=.081 
0.75 0.32 (.21;.44) 0.058 0.003 37 38 Q=338.4; d37; p<.001; /2=89; r2=.090 
Panel B: without outlier 

0.50 0.27 (16; .38) 0.056 0.003 36 37 Q=233.3; dF36; p<.001; /2=84.5; 12=.072 
0.75 0.28 (.17;.39) 0.055 0.003 36 37 Q=288.0; dH=36; p<.001; /2=87.5; 12=.076 
Panel C: with outlier Winzorised 

0.50 0.30 (.19;.41) 0.057 0.003 37 38 Q=246.23; dF37; p<.001; !2=84.9; 12=.076 
0.75 0.30 (.19;.41) 0.055 0.003 37 38 Q=301.3; dF37; p<.001; /#=87.7; 12=.078 


Finally, as stated in the protocol, we ran a sensitivity analysis to test differences between 
published and unpublished reports. The 20 independent effect sizes reported in 19 peer- 
reviewed journals yield a SMD=.31 (95%CI .17 to .45) whereas the 18 effect sizes extracted 
from unpublished reports reported a SMD=.29 (95%CI .11 to .47). The between effect 
difference was not statistically significant (Q=.028; df=1; p>.05). 
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5. Discussion 


5.1 SUMMARY OF MAIN RESULT 


In the following paragraphs we present a summary of our results based on the research 
questions leading this review. 


e Doschool-based programmes reduce the use of exclusionary sanctions in 
schools? 


The analyses reported in previous chapters suggest that school-based interventions are 
capable of producing a small and significant (SMD=.30; 95% CI .20 to .41; p<.001) drop in 
exclusion rates. It means that those participating in interventions are less likely to be 
excluded than those allocated to control/placebo groups. These results are based on 
measures of impact collected on average, six months after treatment. When the impact was 
tested in the long-term (i.e., 12 or more months after treatment), the effect of interventions 
was not sustained. In fact, the impact of school-based programmes showed a substantial 
reduction (50%), and was no longer statistically significant. 


We ran a further analysis including only the seven studies which reported post-treatment and 
follow-up measures. The overall impact of school-based interventions at post-treatment was 
SMD=.21 (95%CI .11 to .30). Even if the impact was slightly reduced, when we included those 
seven studies the average time involved under “post treatment” was 12 months on average, 
meaning that this measure of impact would be more precise and the impact would last 
longer. Coherent with the original results, the effect produced by combining the seven studies 
only was null and non-significant in the long term (i.e., follow-up measures). 


In addition, our results suggest that school-based interventions present different levels of 
impact depending on the type of exclusion. After running moderator analysis by type of 
exclusion, the evidence across studies demonstrated that school-based interventions are 
more effective at reducing in-school than out-of-school exclusion. The impact of programmes 
in the latter case was close to zero with p>.05, meaning that the result could have arisen by 
chance. Moreover, a larger impact was observed in the sanction expulsion, with a moderate 
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and significant effect size (SMD=.53; 95%CI .07 to .98; p<. 05). However, these results must 
be cautiously interpreted due to the low number of manuscripts involved in the calculations. 


We found 28 studies lacking an operational definition of the dependent variable. The studies 
did not specify the type of exclusion or exclusion tested, and in that scenario, we decided not 
to assume this general measure was out-of-school exclusion. The impact of school-based 
interventions in those “general suspensions” was similar to the overall impact, which 
probably suggests that these studies are simply a subset of the overall measure. 


As originally planned, in this review we ran independent meta-analysis testing the impact of 
school-based interventions on a secondary outcome, that is, internalising and externalising 
behaviours. It was hypothesised that a reduction in exclusion would be linked with variations 
in students’ behaviours. 


In the case of externalising behaviours, we were not able to calculate a pooled effect size. 
Results suggest that only five out of the 37 included studies reported a measure for 
internalising behaviours. However, the narrative description suggests that school-based 
intervention had a small effect on the reduction of the above symptoms. 


We ran a meta-analysis based on 15 effect sizes reporting the impact of school based 
intervention on externalising behaviours. Results show negative and non-significant impact 
(SMD=- .005; 95%CI -.09 to .086; p>.05). It therefore follows that interventions aimed at 
reducing exclusion do not necessarily reduce antisocial behaviour. This could be in line with 
evidence suggesting that changes in school policies, rather than changes in behaviour, 
produce a reduction in disciplinary exclusion (Noltemeyer & Fenning, 2013; Skiba et al., 
2015). Two questions arise from these results. First, how do we interpret them, and second, is 
it still worth implementing programmes that show no impact on antisocial behaviour? 


Regarding the first question, one could hypothesise that if race, or more precisely being a 
racial minority in a given country, is a stronger predictor of school exclusion than other 
demographic and behavioural characteristics (as shown in Section 1.2), antisocial behaviour 
would not necessarily explain the rates of exclusion. What is more, it would not be the main 
cause of this punishment being used. A second possibility is that the behaviour outcome was 
not measured properly. 


The second question considers whether the school-based interventions tested in this review 
are still worth implementing if they do not reduce antisocial behaviour. The evidence 
produced by the present review shows that those programmes presenting positive results do 
not report negative side effects, in the sense that control groups do not increase the rate of 
punishment. If that is the case, we suggest that these efforts should be continued. However, 
based on our data, these ideas cannot yet be regarded as conclusive and more research is 
needed about the causes that explain the phenomenon of exclusion. 
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e Are some school-based approaches more effective than others in reducing 
exclusionary sanctions? 


The nine different types of interventions presented in the set of included studies were 
compared to test the hypothesis that some are more effective than others. There was a 
considerable variability across the programmes in overall effectiveness. Enhancement of 
academic skills, mentoring/monitoring, skills training for teachers and counselling/mental 
health services reported the largest and most significant effect sizes. Based on the number of 
studies included in each sub-type, we believe that skills training for teachers and 
mentoring/monitoring represent the stronger and more reliable findings. The subgroup 
analysis reported that differences across types of programmes were significant, implying that 
effects vary depending on the programme implemented. 


In line with previous findings, Tolan et al. (2008) examined 39 experimental and quasi- 
experimental studies and specifically analysed the effectiveness of mentoring in reducing 
delinquency, aggression and drug use, and improving academic achievements. The largest 
positive effects were found for aggression (SMD = .40; 95%; CI = 0.06 to 0.74) and 
delinquency (SMD = .23; 95% CI .11 to .36). The meta-analysis conducted by Eby, Allen, 
Evans, Ng, and Dubois (2008) also found that mentoring programmes had a small and 
significant positive effect on academic performance. 


Overall, mentoring programmes seem to be an effective strategy for reducing violence and 
conduct problems during adolescence. However, on average the effect sizes reported in 
relevant studies are small. In this respect, it is important to bear in mind the findings by 
Tolan et al. (2008). This study suggests that mentoring was more effective when (a) 
participants had greater pre-existing behavioural problems or had been exposed to 
significant levels of environmental risk, (b) they were male, (c) the educational or 
occupational backgrounds of the mentors fitted the goals of the program, (d) mentors and 
youths were successfully paired, with similar interests, and (e) programmes were structured 
to support mentors. 


As far as teachers’ skills for managing students’ behaviour is concerned, previous evidence 
suggests that such programmes can improve teachers’ general instructional and behavioural 
management skills in planning, implementing and maintaining effective classroom practices. 
In fact, a meta-analysis conducted by Gottfredson, Wilson, and Najaka (2002) found that 
classroom or instructional management programmes (k=25) demonstrated a small and 
significant effect in reducing antisocial/aggressive behaviour (SMD= .13, p<.05). The most 
recent meta-analysis carried out by Oliver et al. (2011), concluded that classroom 
management practices (N=12 studies) had a significant, positive effect on reducing problem 
behaviour. Students taking part in the intervention display less disruptive, inappropriate and 
aggressive behaviour in the classroom compared to those in control classrooms. The overall 
effect of the intervention was g= .22; p<.05 (Oliver et al., 2011). Evidence also suggests that 
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programmes with the most positive effects tend to be of a longer duration and tend to 
combine classroom and instructional management strategies with some other major 
ingredient (e.g. parent training or social skills instruction) (Gottfredson et al., 2002). 


The positive relationship between teachers and students has been found to be a factor in 
promoting more prosocial and less aggressive behaviours later in life. A recent study (Obsuth 
et al., 2016; p. 16), using a non-bipartite propensity score matching technique, found that 
“teachers who reported having a more positive relationship with a student at age ten 
observed significantly fewer aggressive and defiant behaviours and more prosocial 
behaviours in the same student concurrently and one year later, at age 11. This was also 
associated with more prosocial behaviours two years later, at age 12 and also with less 
aggressive behaviour at age 13. Similarly, students who perceived a more positive 
relationship with their teacher at age 11 reported fewer aggressive behaviours and more 
prosocial behaviours concurrently and fewer aggressive behaviours two and four years later, 
at ages 13 and 15.” All these results make us believe that investing in teachers’ skills and 
positive relationships between students and teachers is worthwhile, with schools becoming 
target locations for preventing crime and promoting positive psychosocial development. 


Finally, it is worth mentioning that we grouped interventions by those targeting a change at 
student level versus those expecting a change at school level. Even if the programmes 
targeting a change at student level display a larger effect size, the between-group analyses do 
not allow us to conclude that the differences between these intervention strategies are 
statistically significant. 


e Do participants’ characteristics (e.g., age, gender, ethnicity) affect the 
impact of school-based programmes on exclusionary sanctions in schools? 


Differences in post-intervention effects were non-significant when we compared schools 
whose population was predominantly masculine versus mixed schools. Thus, based on our 
findings, we could not confirm the hypothesis that the effect differs by the distribution of 
gender in schools. The same occurred with ethnicity and age. As both variables were 
continuous, we included them in our meta-regression. Based on the available data, we could 
not confirm the hypothesis that the impact of the interventions varies when the intervention 
was implemented with younger or older students. Similarly, variation in the percentage of 
white students did not play a role in explaining the heterogeneity of the effect. 


e Docharacteristics of the interventions, implementation and methodology 
affect the impact of school-based programmes on exclusionary sanctions in 
schools? 


Twenty-five studies (reporting 26 independent effect sizes) stated the presence of training 
hours before the intervention was delivered. In the remaining 12 studies, the authors did not 
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mention any kind of training. When comparing the effect of both sets of studies, the between- 
group analysis was non-significant, meaning that there is no evidence that effect differs by 
presence/absence of training before implementation. 


We ran an analysis to investigate the differences between those interventions that deployed 
monitoring during implementation and those which lacked monitoring. Interventions 
without monitoring produced significantly larger effects than those with it. In other words, 
programmes lacking monitoring produced larger effect sizes. This result is in line with 
previous findings indicating that a lack of monitoring, for instance in routine trials, tends to 
show smaller effects than demonstration evaluations (Farrington, Ttofi, & Ldsel, 2016). 


On the other hand, we observed that interventions run by independent evaluators found 
significantly smaller effect sizes when compared to those studies carried out by researchers 
involved in the design and/or delivery of the programme. Again, this is not surprising. 
Previous research has called attention to this phenomenon as well as the advantages of 
running impendent trials for producing more precise and realistic results (e.g., Eisner, 2009; 
Manuel Eisner & Humphreys, 2012; Lésel & Beelmann, 2006b; Petrosino & Soydan, 2005; 
Wilson, Lipsey, & Soydan, 2003). 


5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE 


In the present review we screened a total of 42,749 citations across 27 electronic databases. 
Pre-defined searches targeted published and unpublished reports from any country, in any 
language, as long as the abstract was presented in English. Only one study in a language 
other than English was identified by our searches. We purposively conducted electronic 
searches in databases involving manuscripts produced in Latin America and other Spanish 
and Portuguese-speaking countries (e.g., SciELO-Scientific Electronic Library Online). 
However, we did not find any evaluations whose characteristics make them includable in our 
review. 


When compared to other contemporary reports (e.g., Dymnicki, Weissberg, & Henry, 2011; 
Mytton et al., 2006), our searches seem to be comprehensive enough to reduce the risk of 
publications bias. All in all, the extent of our searches and the high percentage of retrieved 
full reports make us confident that we have identified a substantially complete collection of 
the available relevant research. 


However, this systematic review has allowed us to identify some gaps in the literature. First 
of all, studies testing the impact of school-based interventions fail to disentangle the impact 
on different forms of disciplinary exclusion. In future, the availability of data calculating the 
impact of interventions across different types of exclusionary punishment could offer more 
detailed evidence in respect to what works in the field of school discipline. 


Secondly, we did not find includable studies testing the impact of restorative justice 
strategies. One single ongoing study (i.e., Acosta, 2015) was detected by our searches but its 
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results are to be released in 2018. As recently stated by Fronius et al., (2016: 17 and 19), the 
research evidence to support restorative justice in schools is still in a nascent state. In fact, 
the scarce evaluations produced so far rest in pre-post designs lacking control comparison 
groups, which would likely have a serious impact on the internal validity of the empirical 
results. The potential for restorative principles to be applied in school settings needs to be 
explored in more detail. 


Thirdly, we believe that the production of independent, high quality evaluations could 
contribute to more transparent and precise evidence regarding the impact of school-based 
programmes. Previous research shows that demonstration programmes or those where 
designers take part in the evaluation tend to produce larger effect sizes (Beelmann & L6sel, 
2006). 


Fourth, much of the evidence presented in this review has been produced in the United 
States where school safety and exclusion is a salient concern for researchers and policy 
makers alike. As shown in Table 1 of the present report, in a sample of high- and middle- 
income countries, exclusion is a widespread school punishment. In European countries, 
exclusion is a more regulated sanction, while in some Latin-American societies exclusion is 
under the discretional decision of each particular school (e.g., Chile). We think that research 
on the use of exclusion in those contexts needs more development. 


Fifth, we observe a lack of research testing the long-term impact of school-based 
interventions. The evidence shows that the impact immediately after intervention tends to be 
larger than that measured several months later (Farrington, Ttofi, & Lésel, 2016). Since in 
real life policy makers need to invest in effective programmes whose results endure long- 
term, we think that an effort should be made to test the impact at least 12 months after 
intervention ends. 


Finally, although we did not include cost-benefit analysis in our aims, data was coded 
regarding the presence/absence of interventions’ costs. Two of our included studies reported 
the economic cost of programme implementation. We believe that cost-benefit analyses are 
key for alerting decision makers about the advantages of early intervention. The prevention 
of exclusion and expulsion from school can reduce future social exclusion, violence and other 
negative correlates described in the research (Noltemeyer, Ward, & Mcloughlin, 2015). 
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5.3 QUALITY OF THE EVIDENCE 


The present review involves only randomised controlled trials. They are considered the best 
methodological design for isolating confounding factors and producing an accurate measure 
of intervention effects. However, as stated earlier in the results section, our included reports 
are lacking a considerable amount of information for judging the quality of the procedures 
carried out. 


More than 50% of the reports fail to provide enough data to judge the precision of 
randomisation, that is, sequence generation and allocation concealment. Likewise, 30% of 
the studies did not report data on the equivalence of the groups after randomisation. As 
recently stated by Roberts & Ker, (2016), missing data on those details can drive analysts to 
identify “false positive” RCTs. We cannot claim that was the case among our studies, but 
clearly the absence of detail imposes serious limitations on the assessment of quality in 
regard to potential threats to internal validity. 


A similar lack of data affected the evaluation of blind outcome measures. Even if most of the 
exclusion measures in our report are based on official records, we cannot ignore the fact that 
teachers or school staff are in charge of imposing sanctions and could potentially be aware of 
a student’s participation in the experiment. More than 50% of the studies present few details 
for judging the level of risk involved in this criterion. 


Randomised controlled trials of educational programmes are receptive to contamination 
(Hutchison & Styles, 2010; Simmons et al., 2015). As participants in the control arm may be 
surrounded by those receiving the treatment, because they share the same school, the 
likelihood of contamination is a matter of concern. Some of the effects of contamination have 
been identified by previous research (Giraudeau & Ravaud, 2009; Torgerson, 2001). 


Contamination may reduce the effect of the intervention, leading to a type II error — that is, 
incorrectly retaining a false null hypothesis (a "false negative") or, described in a different 
way, a “rejection of an effective intervention as ineffective because the observed effect size 
was neither statistically nor clinically significant” (Torgerson, 2001). In the present meta- 
analysis, 76% of the studies suggest a high risk of contamination bias. It could be the case 
that our estimation of the impact is underestimated. 


In future research, it seems that the use of cluster randomised control trials would help 
ameliorate these biases. Even if clustered RCTs involve complex designs and demand huge 
efforts, the quality of the much-needed evidence can be enhanced by using this 
methodological design (Hutchison & Styles, 2010). However, the randomisation of clusters 
implies challenges for researchers. “The main issue is that observations from the same cluster 
are more similar than observations from two different clusters. This situation requires the 
use of both an inflated sample size and adapted statistical analysis to take into account this 
concern” (Giraudeau & Ravaud, 2009, p. 1). 
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5.4 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW 
PROCESS 


There are several limitations that could affect the results of the present review. It is 
important to acknowledge that even though we focused on randomised controlled trials, 
which are supposed to be the best evidence for measuring evaluation impact, included 
studies present limited information for judging quality bias. 


Some 35% of the included studies reported results based on samples with less than 100 
participants. The small size of the samples involved in some of the primary research could 
impose clear limitations on the ability to estimate the effects of interventions. 


As reported in the moderator analysis, the independence of the evaluator explained the 
heterogeneity of effect sizes. Even if independent teams reported on a good number of our 
studies, close to 50% of them did not use independent evaluators. This fact could add some 
bias in to our results. 


However, it is also important to elucidate some of the advantages of the present study. First 
of all, this systematic review and meta-analysis is the first attempt to collect and statistically 
summarise interventions pursuing a reduction in school exclusion. As such, we believe that 
our report offers an overview of the amount, characteristics, limitations, and quality of the 
available evidence, as well as a measure of the size of the effect achieved by different types of 
intervention. The reader can find in this report an updated review of the evidence produced 
in the United States and the United Kingdom. 


Secondly, we have endeavoured to use an exhaustive coding process to provide careful 
descriptions of the studies as well as a meticulous analysis of the statistical data available. 
Due to the inclusion of cluster data, corrections were introduced to make the information 
comparable at an individual level. Such corrections were carried out using the most recent 
strategies suggested by Hedges, (2007) and Pigott, (n.d.). We therefore believe that our 
calculations are at a low risk of underestimating the size of standard errors. 


5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES 
OR REVIEWS 


The present review is the first meta-analytical effort to identify the impact of school-based 
intervention at reducing school exclusion. In that sense, we have no similar evidence with 
which to contrast our findings. 


However, the results provided by our review seem coherent with other studies testing the 
impact of school-based intervention on behavioural outcomes. For instance, Wilson et al., 
(2011) conducted a meta-analysis on school-based programmes looking for a reduction in 
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drop-outs. Our results more or less target similar programmes and the detected effect size 
follows a similar trend. In the same vein, Mytton et al., (2006) conducted another review 
testing the impact of school-based interventions in preventing violence. They found similar 
problems with quality assessment and reported an overall effect size slightly better than the 
one produced by the present review. However, in that last case, the long-term effects are 
statistically stronger and last longer. The reduction in aggression reported by Mytton et al., 
(2006) is not clear in our results. 


In a more general examination, a recent review of reviews conducted by Farrington, Ttofi, & 
Losel, (2016) found a mean effect33 of school-based programmes equal to SMD=.184 (95%CI 
.16 to .20). Across the studies, effect sizes ranged between SMD=.091 to SMD=.631. Our 
findings are, therefore, more or less coherent with the impact of interventions in school 
settings evaluated by previous meta-analysis. 


33 The original effect sizes are presented in Fig. 2.1, page 63. They are reported in OR; we have converted those 
values to make it more comparable with the metric we use in the present review. 
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6. Authors’ conclusions 


The empirical evidence produced by this report suggests that non-punitive school-based 
programmes can reduce the use of exclusion. Even if the effects are not sustained for the long 
term, data shows that in the short term (i.e., six months on average) schools can opt for 
different and more effective approaches to managing discipline, rather than zero-tolerance 
policies. This review, aimed at testing the effectiveness of school-based programmes, offers a 
broad overview not only of the effectiveness of different interventions, but also uncovering 
findings that can guide public policy and future research. 


6.1 IMPLICATIONS FOR PRACTICE AND POLICY 


Research consistently reveals that school exclusion is disproportionately used as a 
punishment for ethnic minorities, males and those coming from low socio-economic 
backgrounds. Exclusion as a disciplinary school measure seems to reduce school attendance, 
increase drop-out rates and restrict future possibilities for inclusion in the labour market. 
Aside from that, research has also been consistent in finding a correlation between 
permanent exclusion and antisocial behaviour. 


Though the causal link between exclusion and the above outcomes is still not clearly 
established, research based on observational data suggests that the consequences of 
exclusion affect not only students and their families but extend to the whole society. Indeed, 
as stated by Rumberger and Losen (2016), exclusion may involve a high economic cost to the 
taxpayer. In their most recent report, the authors provide a conservative estimate of the 
economic impact of exclusion. They assert that, in the United States, tenth grade exclusions 
alone account for more than 67,000 school drop-outs, at a cost of $35 billion.34 


Although our results must be cautiously considered, the evidence produced by this review 
suggests that school administrators and policymakers do have alternatives to exclusion when 


34 To calculate the impact of exclusion on dropout rates, the study used longitudinal data collected via surveys and 
standardised tests to observe the percentage of students who graduated on time, two years after the expected date 
of graduation, and four years after. The independent variable was exclusion and the study involved independent 
controls such as demographic variables, family variables and school performance variables. All in all, the study 
showed that “in the U.S., only 71 percent of tenth graders who received a suspension graduated from high school, 
compared to 94 percent of tenth graders who did not receive a suspension. In other words, being suspended [was] 
associated with a 23 percentage-point decrease in the likelihood of graduating” (Rumberger & Losen, 2016; p.12). 
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dealing with disciplinary problems. In our findings, prevention strategies have a small but 
encouraging impact on exclusion rates, at least in the short term. When comparing different 
types of programmes, it appears that prevention programmes targeting teachers’ skills as well 
as those introducing mentoring/monitoring schemes can have a positive impact in reducing 
exclusion. These results are in line with previous research and also with previous narrative 
reviews which emphasize the importance of teachers’ skills and mentoring programmes in 
promoting prosocial behaviours and values (Freiberg & Lapointe, 2006; Gottfredson et al., 
2002; Oliver et al., 2011; Tolan et al., 2008). 


It is important to clarify that this is most applicable to the United States, where the majority 
of the assessed evidence was collected. As expressed in the previous paragraphs, some flaws 
affect the contemporary primary evaluations present in this review. However, these results 
should encourage researchers to produce better quality evidence rather than abandoning 
their efforts to find strategies to replace exclusionary punishments. 


6.2 IMPLICATIONS FOR RESEARCH 


Clearly, more primary research evaluating the impact of prevention programmes targeting 
school discipline is needed. Ideally this research should be conducted under high 
methodological standards, accounting for mediating mechanisms that lead to a reduction in 
exclusion rates. 


In particular, the results of this first review on the topic identify some implications for future 
research, laid out below: 


e Addressing the racial gap. Most of the literature reviewed in the present study 
indicates that racial or ethnic identity plays a central role in predicting school 
exclusion, even after controlling for demographic and behavioural variables. More 
research needs to be developed for testing the mechanisms that produce different 
treatment for some racial minorities, such as African-American and Latino students 
in the United States or Black Caribbean and Gypsy/Romany students in the United 
Kingdom. It is necessary to understand the individual and social processes that lead 
to the overrepresentation of minorities in school exclusion. As suggested by some 
scholars, the divergence between students and teachers’ cultural expectations could 
potentially contribute to misinterpretation of each other’s behaviour, fears and 
conflicts (Gregory et al., 2010). Understanding the mechanisms that make race a 
predictor of exclusion could have implications for future policy and practice. For 
instance, as stated in previous research, greater diversity among staff could be helpful 
in promoting understanding of cultural differences and reducing bias (Irvin, 2002). 


e Causal effects of exclusion. The present research has addressed the lack of 
certainty about the genuine causal effects of exclusion. A review of previous research 
suggests that exclusion could be a risk factor for a set of negative outcomes already 
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described in the above paragraphs. At the same time, an overview of previous 
criminological theories suggests that punishment could present different effects on 
different individuals. The truth is that the causal effect of punishment on students’ 
behaviour is still a long way from being fully understood. If school exclusion is simply 
a marker of an underlying antisocial syndrome, it could be beneficial to invest in 
prevention programmes targeting, for example, deviant behaviours (e.g., violence, 
drug use, crime, abuse, and neglect) and personality features (e.g., aggressiveness, 
lack of empathy, lack of remorse) associated with antisocial syndrome (Farrington, 
2003). However, if school exclusion is proved to be the cause of detrimental outcomes 
later in life, it will be worth investing in more programmes focusing specifically on the 
reduction of school exclusion. 


Mediating mechanisms that explain reductions/increases in exclusion 
rates. Future evaluations of school-based interventions aimed at reducing exclusion 
need to explore the presumed causal mechanisms that lead to that reduction. More 
theoretically informed trials that examine causal mechanisms are important to design 
better interventions. In fact, this information would be crucial in planning future 
prevention programmes since causal mechanisms can shed light on what works for 
whom and under what conditions. 


Recognise the key role of statistical power. Future research could benefit from 
prospective power analysis. That means that, during the early stages of the 
evaluation’s design, there must be consideration given to the sample size required to 
detect an effect. This early effort can protect studies from “underpowered or 
wastefully overpowered” samples which can affect outcomes by Type I and Type II 
errors (Ellis, 2010). Future meta-analysis should also probably give more attention to 
this issue and, as suggested by Farrington et al., (2016), it seems advantageous to set 
a minimum sample size for inclusion in reviews. 


Tackle the challenges of running randomised controlled trials in school 
settings. Research conducted within schools often struggles to isolate experimental 
and control groups, with the subsequent risk of contamination. As stated earlier in 
this report, the use of cluster randomised controlled trials can present an opportunity 
for tackling this when conducting research in educational settings. 


Attempt blind assessment in school-based randomised controlled trials. 
The characteristics of schools make it hard to blind all those involved in trials. 
Teachers, students and school counsellors are likely to be involved in the experiment, 
making it difficult to control the social desirability effect of those participating in the 
study (Hutchison & Styles, 2010). This challenge needs to be addressed by future 
studies by at least blinding those who collect data. It also seems necessary, at the level 
of meta-analytical studies, that the tools used to measure quality bias in school 
contexts be adapted. In our experience, some of the available “risk of bias tools” 
seemed more suited for medical trials than for school-based experiments. 
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Risk of quality bias. Meta-analysis integrates the quantitative evidence from 
different but related studies to summarise a whole body of knowledge on a research 
question. As in the present review, meta-analysis can answer the question about the 
effectiveness of a given intervention. By the fact that meta-analysis combines results 
from different primary research reports, quality bias involved in primary research can 
jeopardise the validity of the “meta” results. In brief, quality bias refers to systematic 
error, “meaning that multiple replications of the same study will produce different effect 
estimates because of sampling variation even if they would give the right answer on 
average” (Higgins & Green, 2011, p.188). 


As stated by Higgins & Green (2011, p. 189) “more rigorous studies are more likely to 
yield results that are closer to the truth”. Differences in the quality of performed studies 
can result in false positive conclusions when less rigorous studies are biased toward 
overestimating an intervention’s effect. They can also arrive to false negative conclusions 
in those cases where less rigorous studies are inclined towards underestimating an effect. 


To evaluate risk of quality bias, the present study involves an evaluation of each of the 
included reports. Three categories were used to judge a study report; i) low risk; ii) 
unclear risk and iii) high risk of bias. When it came to randomisation, we found a small 
percentage of high-risk cases. However, it is important to note that a large proportion 
of studies (59%) were categorised as “unclear risk”. This is because many reports 
presented succinct descriptions of the randomisation process without detailing the 
methods of sequence generation for instance. When we tested the quality of allocation 
procedures, once again, we found a high percentage of studies (54%) reporting 
minimal details of allocation concealment. Similar findings were observed for 
baseline equivalence (22% of assessed studies reported limited information), attrition 
(30% unclear risk) and blindness of the outcome assessment (54%). These findings do 
not necessarily mean that studies present quality bias, they just inform us that the 
authors omitted data which is relevant to interpret the results. Future research would 
benefit from following CONSORT standards. Registration of trials, even if they are 
PhD theses, would represent a huge benefit not only for the scientific community, but 
also for those interested in evidence-based decision making. 


Cross-cultural research. Based on our findings, the evidence so far is largely 
coming from the United States. More research needs to be done in other countries 
where school exclusion is an issue. We know that evidence suggesting effective 
approaches in some countries/cultures will not necessarily have the same 
effectiveness when translated to different populations. Those making decisions about 
how to reduce exclusion in their own country need to have access to detailed 
information addressing their particular needs. 


Innovative strategies. More research needs to be conducted on innovative 
strategies, for example those involving empathy-based philosophies. This was the 
basis for the intervention tested by Okonofua et al. (2016, p.5521) included in this 
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review. The intervention is focused on encouraging teachers to adopt an empathetic 
attitude towards discipline; it is low cost and demonstrates long-term effects (rates 
from 9.6% to 4.8%). As stated by the authors, “teachers’ mind-sets about discipline 
directly affect the quality of teacher-student relationships and student exclusions and, 
moreover, can be changed through scalable intervention”. 


Restorative justice programmes. In addition to the above, the debate about 
strategies for reducing exclusion in schools have recently been enriched by some 
research suggesting that restorative justice programmes could have a promising 
impact (Anyon, Gregory, Stone, Farrar, & Downing, 2016; Fronius et al., 2016). 
Scholars have highlighted the advantages of restorative justice over exclusion by 
showing that these programmes focus on building peaceful and empathetic 
relationships, ask for the involvement of all parties in achieving conflict resolution, 
look at the harm done to those affected by misbehaviour, and promote reintegration 
rather than exclusion (Drewery, 2004; Hostetler, 2014; Varnham, 2005). However, to 
date, there is no clear evidence about its effectiveness (Fronius et al., 2016). Empirical 
research needs to be conducted testing the impact of restorative justice in school 
settings. Two challenges need to be addressed here. First, restorative programmes 
need to be implemented following the guidelines and characteristics of the restorative 
processes. This is not a simple task for schools used to dealing with conflict in a 
punitive way. Secondly, randomised controlled trials or well-controlled studies need 
to test the impact of restorative justice practices in dealing with conflict inside 
schools, such as bullying or other forms of violence (Sherman & Strang, 2007). A 
potential impact in the reduction of conflict could lead to a reduction in the use of 
exclusion or other exclusionary strategies. 
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9. 


9.1 


Tables 


CHARACTERISTICS OF INCLUDED STUDIES 


The following tables offer a succinct characterisation of each included study. We selected 11 


characteristics from our coding which will allow the reader an overview of the study. 


Qn 


10. 
11. 


12. 


142 


Methodological design, clarifying the type of randomised controlled trial (e.g., cluster 
randomised, matched pairs randomisation). 

Characteristics of the participants, involving the size of the sample during 
randomisation, school grade of participants, percentage of males and predominant 
ethnicity. 

Location of the study, detailing the city and country where the study was 
implemented. 

Brief description of the intervention. 

Programme deliverers, those in charge of implementing the programme. 

Evaluator role. When the evaluator is only in charge of conducting the trial, we define 
that as “an independent evaluator.” If in addition the researcher delivers or designs 
the programme, we report that additional role. 

Outcomes measured in the study, not only the outcomes we are interested in. 

Length of the intervention. 

Assessment. This data refers to the measures carried out during the evaluation; for 
instance, baseline and post treatment, or baseline, post treatment and 12 month 
follow up. 

Attrition. This data provides an estimate of the percentage of reported attrition. 
Conflict of Interest Statement (COI). Offers information on the absence/presence of a 
formal and explicit statement declaring conflict of personal or institutional interest. 
Potential Conflict of Financial Interest (CoFI) 
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1) Allen, Philliber, Herrling, & Kuperminc (1997) 


Methods Randomised controlled trial 
Participants 695 ninth through twelfth grade school students 
Treatment: 14% Male, 68% Black 
Control: 17% Male, 67% Black 
Location United States (nationwide) 
Interventions Teen Outreach Program. It is a volunteer service programme, designed to prevent teen 


pregnancy and academic failure by enhancing normative processes of social 
development. The intervention involves supervised volunteer community service, 
classroom-based discussion of service experience and classroom-based discussions 
related to social development tasks of adolescence. 


Programme deliverers 


Teachers 


Evaluator role 


Outcomes measured 


Length of intervention 


Assessment 


Unclear 


School suspension 
Teen pregnancy 
Course failure 


One academic year (around 35 weeks) 


Baseline and post treatment 


Attrition 


Treatment group= 5.3% 
Control group= 8.4% 


COl statement 


Not presented 


CoFl Unclear 
2) Arter (2005) 
Methods Randomised controlled trial 
Participants 52 sixth grade school students 
100% Male, 72.5% African-American 
Location Maryland, US 
Interventions Positive Alternative Learning Support (PALS). The programme aims to make the student 


more competent in the school environment by providing integrated behavioural support, 
academic support, group counselling and mentoring. 


Programme deliverers 


School administrator, school psychologist, counsellors, teachers, special education 
teachers, one volunteer parent and the researcher 


Evaluator Role 


Delivered the programme 


Outcomes measured 


Attendance 

Academic achievement 
Office referrals 
Suspension from school 


Length of the intervention 


18 weeks 


Assessment 


Baseline and post treatment (immediately after intervention) 


Attrition 


24% 


COl statement 


Not presented 


CoFl Possible. The author delivered the intervention although the programme does not appear 
to be commercially available (p. 39) 
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3) Barnes, Bauza, & Treiber (2003) 


Methods Randomised controlled trial 
Participants 45 high school students 
71% Male, 100% African American 
Location Richmond County, United States 
Interventions Stress reduction via Transcendental Meditation. 
Programme deliverers Certified instructor 
Evaluator role Independent 
Outcomes Tardy periods 
Absentee periods 
Grades 
Rule infractions 
Days suspended 
Length of intervention 12 weeks 
Assessment Baseline and during treatment 
Attrition 9% 
COl statement Declared 
CoFl Unlikely, independent evaluator. 
4) Berlanga (2004) 
Methods Randomised controlled trial 
Participants 80 eighth grade school students 
Gender is not clearly reported (mixed). 80% Hispanic 
Location South Texas, United States 
Interventions Grades, Attendance and Behaviour (GAB). In this programme, a cognitive-behavioural 


classroom guidance curriculum is combined with supportive and individualized solution- 
focused counselling sessions (six sessions). 


Programme deliverer School counsellor 
Evaluator role Independent 
Outcomes Self-esteem 
Self-conception 
Academic achievements 
Attendance 
Office referrals 
Suspensions 
Length 12 weeks 
Assessment Baseline and post treatment 
Attrition 21% 
COl statement Not presented 
CoFl Unlikely, independent evaluator (p. 135-36) 
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5) Bradshaw, Waasdorp, & Leaf (2012) 


Methods Clustered randomised controlled trial 
Participants 12,334 students nested in 37 elementary schools 
52% Male, 46.1% White (largest ethnic group) 
Location Maryland, United States 
Interventions School-Wide Positive Behavioural Interventions and Supports (SWPBIS): a programme that 


Programme deliverers 


Evaluator role 


targets a systemic change process in a whole school or in a district. SWPBIS aims to 
reduce students’ behaviour problems by changing staff behaviours and developing systems 
and supports to meet children’s behavioural needs. 


School staff, teachers and administrators 


Independent 


Outcomes 


Aggressive and disruptive behaviour 
Concentration problems 

Pro-social behaviour 

Emotion regulation 

Office disciplinary referrals 
Out-of-school suspension 


Length 


Assessment 


Four years of implementation 


Five times over the course of four years. 


Attrition 


COl statement 


“Participation rate was consistently high, [...] no evidence that missing data was 
problematic” p. e1140 


Declared 


CoFl 


Unlikely, independent evaluator (p. e1136) 


6) Bragdon (2010) 


Methods 


Randomised controlled trial 


Participants 68 eighth-grade students 
61% Male; 53% African-American 
Location United States 
Interventions Teach team project. The intervention is a multi-component drop-out prevention involving 


career exploration, a career awareness course and daily check-in/check-out monitoring by a 
school counsellor or a mentor teacher. 


Programme deliverers 


Counsellor and mentor teachers 


Evaluator role Independent 

Outcomes Academic performance in Mathematics and Language 
Attendance 
Disciplinary referrals 
Suspensions 

Length Nine weeks 

Assessment Baseline, post treatment and follow-up 

Attrition Results suggest no attrition 


COl statement 


Not presented 


CoFl Possible. Manuscript suggests that the researcher adapted an intervention based on 
previous evidence (p. 7-15). 
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7) Brett (1993) 


Methods Randomised controlled trial (randomisation of classrooms N=6) 
Participants 126 seventh grade students. 
Mixed gender (percentages not given), 100% African-American 
Location District of Columbia, United States 
Interventions Efficacy, DC is a mentoring programme aimed at motivating and assisting students to excel 


academically and socially. 


Programme deliverers 


Eight adult volunteer mentors 


Evaluator role Independent 
Outcomes Absenteeism 
School suspension 
Self-esteem 
Academic achievements 
Attitudes towards learning 
Duration 12 weeks 
Assessment Baseline and post-treatment (immediately after the intervention) 
Attrition 23% 


COI statement 


Not presented 


CoFl 


Unlikely. The study author is not the programme developer, collaborator with the 
programme developer or licence holder (p.110) 


8) Burcham (2002) 


Methods 


Randomised controlled trial 


Participants 71 seventh and eighth grade pupils in two schools 
56% Male, ethnicity not reported 
Location Kentucky, United States 
Interventions Social Problem Solving Skills Training. Framed in a cognitive behavioural model, the 


intervention “taught children problem solving skills using modelling, guided practice or 
applied practice”. It is a manualised intervention of 27 lessons. 


Programme deliverer 


Two school psychologists, one of them the evaluator 


Evaluator role 


Delivered the programme 


Outcomes 


Social and academic competency 
Self-concept 

In-school and out-of-school suspension 
Disciplinary referrals 

School tardies 


Length of intervention 


30 weeks 


Assessment 


Baseline, post treatment (immediately after intervention) and follow-up (18 months later) 


Attrition 


One students missed at post test 
43% at follow up 


COI statement 


Not presented 


CoFl Possible. Researcher seems to be involved in programme development (p. 42) and also 
delivered part of the intervention (p.53-4) 
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9) Collier (2002) 
Methods 


Randomised controlled trial 


Participants 60 elementary school students 
100% Male, 100% African-American 
Location Washington, US 
Interventions Pro-social skills training. Skills involve anger management, the development of 


interpersonal skills and problem solving techniques. 


Programme deliverer 


External psychologist 


Evaluator role 


Delivered the programme 


Outcomes Oppositional behaviour 
Cognitive problems 
Hyperactivity 
ADHD 
School suspension 
Length Eight weeks 
Assessment Baseline and post treatment 
Attrition 15% (51 remain) 


COI statement 


Not presented 


CoFl 


Possible. Author delivered and tested the intervention (p.69). Programme seems not to be 
commercially available. 


10) Cook et al. (2014) 


Methods Randomised controlled trial 
Participants 106 ninth to tenth grade school students 
100% Male, 95% Black 
Location Chicago, US 
Interventions Two interventions to be tested. Becoming a Man (BAM) is a non-academic intervention, 


which exposes youth to pro-social adults and provides social-cognitive skill training. The 
Match Model is an academic intervention, providing intensive individualised instruction by 
tutors. 


Programme deliverer 


College educated individuals 


Evaluator role 


Unclear. “Intervention was delivered by staff hired by our own research team” (p.11) 


Outcomes 


Maths achievements 

Reading achievements 

Discipline incidents 

Days absent 

Out-of-school suspension 

Participation in extra-curricular activities 


Length 


BAM: 27 weeks during one academic year 
Match-Model: daily, one academic year 


Assessment 


Baseline and post treatment 


Attrition 


Author recognise attrition. Use of Multiple Imputation and ITT analysis 


COl statement 


Not presented 


CoFl 


Unclear 
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11) Cornell, Allen, & Fan (2012) 


Methods Randomised controlled trial 
Participants 201 school students (K-12) 
73% boys, 73% African-American 
Location Virginia, US 
Intervention Virginia Student Threat Assessment Guidelines is a problem-solving approach to violence 


Programme deliverer 


Evaluator role 


prevention aimed at resolving conflict and working out a solution that allows the student to 


continue in school. 


School administrator, a law enforcement officer or school resource officer, and one or more 


mental health professionals 


Designed the programme 


Outcomes Long-term school suspension 
Access to counselling 
Alternative school placement 
Parent conference 
Victim’s parents notified 
Length Not detailed 
Assessment Baseline and post treatment 
Attrition Results suggest no attrition 


COI statement 


Declared. Main author designed the Virginia student threat assessment guidelines. 


CoFl 


Possible. See p.100, footnote. 


12) Crowder (2001) 


Methods Randomised controlled trial 
Participants 109 seventh grade school students 
45% Male, 99% Black 
Location United States 
Interventions Gang resistance, education and training (GREAT). Intervention involves leadership skills 


development and a character education curriculum. The curriculum involved drug 
resistance, gang resistance, and non-violent confrontational skills among others. 


Programme deliverer 


Police Officer, assisted by two teachers 


Evaluator role 


Independent evaluator 


Outcomes 


Out-of-school suspension 
In-school suspension 

Office referrals 

Tardies 

Unexcused absences 
Students achieving honour roll 


Length 


Nine weeks 


Assessment 


Baseline and post treatment 


Attrition 


Results suggest no attrition 


COl statement 


Not presented 


CoFl Unlikely. Researcher was a PhD student who asked permission to run an evaluation 
implemented by the Metropolitan Police Service (see appendix B). 
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13) Dynarski et al. (2003) 


Methods Randomised controlled trial 
Participants 968 elementary school students 
46.4% Male, 67% Black 
Location United States 
Interventions 21st Century Community Learning is an after-school programme delivered in a public school 


building providing academic support and recreational/cultural activities. 


Programme deliverer Teachers 
Evaluator role Independent 
Outcomes Achievements 
Levels of effort 
School suspension 
Absenteeism 
Tardies 
Parental supervision 
After school activities 
Length One academic year 
Assessment Baseline, post treatment and one-year follow-up 
Attrition 11% 
COl statement Not declared 


CoFl 


Unlikely. See Dynarski et al., 2004, p. XV 


14) Edmunds et al. (2012) 


Methods Randomised controlled trial 
Participants 1607 ninth grade school students 
41.4% Male, 60.2% White 
Location North Carolina, US 
Interventions Early College High School targets students underrepresented in college (i.e., low income, 


the first in their family to go to college or members of a minority group). It is a personalised 
programme designed to boost the academic progress of students, minimising barriers 
between high school and college. 


Program deliverers 


Teachers, school staff. 


Evaluator role Independent 
Outcomes Course taking patterns and success 
Suspension 
Absences 
Planning to attend four-year college 
Algebra | 
College prep. maths courses 
English 
Length Not clearly stated 
Assessment Baseline and post treatment. Unclear how many months or years after intervention. 
Attrition 4.8% 


COl statement 


Not presented 


CoFl 
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15) Farrell, Meyer, & White (2001) 


Methods Clustered randomised controlled trial 
Participants 626 sixth grade school students, nested in 27 classrooms 
50.2% Male, 96% Black 
Location Richmond, Virginia, US 
Interventions Responding in Peaceful and Positive Ways (RIPP) is a universal violence prevention 


program. “The goal of RIPP is to increase adolescents’ capacity and motivation to respond 
to developmental challenges in ways that facilitate social skill acquisition and acceptance of 
personal responsibility” (p.452). 


Programme deliverers 


Evaluator role 


Prevention specialists 


Designed the programme 


Outcomes 


Disciplinary violations for violence 
In-school suspensions 

Out-of-school suspensions 

Violent behaviour frequency 

Drug use frequency 

RIPP knowledge test 

Problem situation inventory 

Attitudes toward nonviolent behaviour 
Attitudes toward violent behaviour 


Length 


25 weeks 


Assessment 


Baseline, post-treatment, six-month follow up and 12-months follow-up. 


Attrition 
COI statement 


Yes. Attrition affected both groups. ITT analysis. 


Not declared 


CoFl 


Possible. See p. 452 


16) Feindler, Marriott, & Iwata (1984) 


Methods Randomised controlled trial 
Participants 36 junior high school students 
Gender and ethnicity not reported 
Location Not reported 
Interventions Anger Control Training is based on a cognitive behavioural model. Training sessions focus 


on the components of the provocation cycle, self-monitoring skills, self-control, problem 
solving, time-out responses, relaxation techniques, and appropriate verbal and non-verbal 
assertive responses. 


Programme deliverers 


Trained therapist and programme aide 


Evaluator role 


Unclear 


Outcomes Problem solving 
Self-control 
School expulsion 
Length Seven weeks 
Assessment Baseline, during treatment and five-week follow-up 
Attrition Results suggest no attrition 
COl statement Not declared 
CoFl Unclear 
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17) Harding (2011) 
Methods 


Randomised controlled trial 


Participants 48 eighth grade school students 
60.4% Male, 97.9% White 
Location East England, United Kingdom. 
Interventions Over to You is a programme based in a cognitive behavioural approach. It encourages 


Programme deliverer 


behavioural change as well as offering training in social skills enhancement (e.g., self- 
awareness, decision-making, empathy, conflict resolution). 


Educational psychologist, trained member of the school staff 


Evaluator role 


Delivered the programme 


Outcomes Emotional literacy 
Behaviour (SDQ) 
School exclusion 

Length Six weeks 

Assessment Six-months follow-up 

Attrition 8% 

COI statement Not declared 


CoFl 


Possible. The researcher implemented the intervention (p. 97-107) 


18) Hawkins, Doueck, & Lishner (1988) 


Methods Randomised controlled trial 
Participants 160 seventh grade school students (low achievers) 
47% Male, ethnicity not reported. 
Location Seattle, US. 
Interventions Proactive Classroom management can be defined as a package of instructional methods 


(interactive teaching and co-operative learning) in mainstream schools. Improved instruction 
is aimed at benefitting low achievers by improving behaviour, attitudes and academic 
results. 


Programme deliverer 


Teachers 


Evaluators role 


Independent 


Outcomes Students social bonding to school 
Students achievements 
Expectations and aspirations for education 
Antisocial behaviour 
Suspension and expulsion from school 

Length One academic year 

Assessment Baseline and post treatment 

Attrition 11% 

COl Statement Not declared 

CoFl Unlikely 
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19) Hirsch, Hedges, Stawicki, & Mekinda (2011) 


Methods Randomised controlled trial 
Participants 535 High school students 
41% Male, 76% African-American 
Location Chicago, United States. 
Interventions After School Matters (ASM) is a programme offering paid apprenticeship-type experiences. 


Programme deliverer 


Evaluator role 


Instructors provide information, guidance and feedback, and introduce students to the 
standards, language and culture of that line of work. The experience helps students begin to 
appreciate and adapt to the culture of the workplace and improve the “soft skills” 
increasingly demanded by employers. Intervention takes place in the host high school. 


Instructor and school members 


Independent 


Outcomes Positive youth development 
Marketable job skills 
Academic outcomes 
Problem behaviour 
School suspension 

Length 20 weeks 

Assessment Baseline and post treatment 

Attrition 18% 

COl statement Not declared 


CoFl 


Unlikely. See acknowledgments section. 


20) Hostetler & Fisher (1997) 


Methods Randomised controlled trial 
Participants 317 fourth grade school students and their families. 
57.4% Male, 33.8% Caucasian 
Location Lancaster, US. 
Interventions Project CARE, a substance abuse prevention program, aimed at improving students’ skills 


(problem solving, peer resistance) and parents’ skills. 


Programme deliverer 


Psychologist and prevention specialist 


Evaluator role 


Independent 


Outcomes Negative behaviours 

Intent to use substances 

Alternative activities 

Communication with parents 

Substance use 

School absences, school suspensions, school grades 
Length One school year plus a summer, one session per week 
Assessment Baseline, post treatment, 12-months follow-up, 21-months follow-up 
Attrition 40% (12 months follow-up) 

75% (21-months follow-up) 
COI statement Not declared 
CoFl Unlikely 
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21) Ialongo, Poduska, Werthamer, & Kellam (2001) 


Methods Clustered randomised controlled trial 
Participants 678 elementary school students, nested in 27 classrooms (nine schools) 
53% Male, 86.8% African-American 
Location Baltimore, US. 
Interventions 1) Classroom centred intervention (CC) is aimed at improving classroom management. It 


Programme deliverers 


involves three main components: curriculum enhancement, enhanced behaviour 
management practices and back-up strategies for children not performing adequately. 
2) Family School Partnership (FSP) is an intervention targeting improvement in parent- 
teacher communication and parents’ child behaviour management strategies. 


CC is delivered by teachers 
FSP is delivered by teachers plus school psychologist or school social worker 


Evaluator role 


Designed the programme 


Outcomes Conduct problems in school 
School suspension 
Academic achievements 
Mental health needs 
Assessment Baseline and five-year follow-up. There was a post treatment measure but school 
suspension was not measured. 
Attrition 24% 
COl statement Not declared 


CoFl Possible. Researcher seems to be the programme designer (p.602) 
22) Johnson (1983) 
Methods Randomised controlled trial 
Participants 60 seventh and eighth grade school students 
Gender and ethnicity not reported 
Location Washington, US 
Interventions Project A.T.T.E.N.D. (Alternatives To Trouble Encouraging New Directions). An education 


programme targeting self-responsibility for maintaining school discipline. An alternative to 
punitive disciplinary methods that combines supervision of attendance, behaviour and 
classwork as well as counselling. 


Program deliverer 


School counsellor 


Evaluator role 


Design and delivery 


Outcomes Attendance (absences and tardies) 
Suspension 
Number of misbehaviour referrals 
Grades 

Length Nine weeks 

Assessment Baseline and post treatment 

Attrition No attrition reported 


COl statement 


Not declared 


CoFl 


Possible. Researcher is programme developer and deliverer (p.7) 
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23) Lewis et al. (2013) 


Methods A matched-pair, clustered randomised controlled trial 
Participants 624 elementary school students, nested in 14 schools 
47% Male, 48% African-American 
Location Chicago, United States 
Interventions Positive Action programme includes a K-12 classroom curriculum involving six components: 


self-concept, social and emotional positive actions for managing oneself responsibly, 
positive actions directed toward physical and mental health, honesty, getting along with 
others, and continually improving oneself. The programme also includes teacher, 
counsellor, family, and community training as well as activities directed toward school-wide 
climate development. 


Programme deliverer 


School facilitators 


Evaluator role 


Design the programme 


Outcomes Support of aggression 
Bullying 
Disruptive behaviours 
Violence 
Disciplinary referrals 
Suspension 
Length One year 
Assessment Baseline (2004), post treatment (2005) plus multiple follow-ups. 
Attrition 5% post test 
COI statement Declared 


CoFl Likely. “The research described herein was done using the program, the training, and 
technical support of Positive Action, Inc., in which Dr. Flay’s spouse holds a substantial 
financial interest. Issues regarding conflict of interest were reported to the relevant 
institutions and appropriately managed following the institutional guidelines.” (p.629) 

24) Mack (2001) 

Methods Randomised controlled trial 

Participants 20 fourth to six grade school students 
50% Male, 100% African-American 

Location Alabama, US 

Interventions ICAN Kids! Control Anger Now and Skills for Living curriculum. A behavioural group 


counselling intervention programme involving techniques such as behaviour contracts, 
relaxation, role-play and modelling. 


Programme deliverer 


Counsellor 


Evaluator role 


Design and delivery 


Outcomes Disciplinary referrals 
School Suspension 
Length Six weeks 
Assessment Baseline, during treatment, post treatment and three weeks follow-up. 
Attrition No attrition reported 


COl statement 


Not presented 


CoFl 
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25) Obsuth et al. (2016) 


Methods Clustered randomised controlled trial (minimisation) 
Participants 738 secondary school students, nested in 36 schools 
71% Male, 40.3% Black-Caribbean/Black-African 
Location London, United Kingdom 
Interventions Engage in Education — London (EiE-L). The intervention targeted their social 


Programme deliverer 


communication and broader social skills with the aim of reducing school exclusions and 
problem behaviours. 


Core worker trained to deliver the intervention 


Evaluator role 


Independent 


Outcomes 


School exclusion 
Interpersonal communication 
Pro-social skills 
Student-teacher relationship 
Antisocial behaviour 

Bullying perpetration 
Delinquency 

Arrests 


Length 


12 weeks 


Assessment 


Baseline and post treatment 


Attrition 


12% (based on official records) 


COl statement 


Declared 


CoFl 


Unlikely. Independent evaluation. 


26) Okonofua, Pauneskua, & Walton (2016) 


Methods Clustered randomised controlled trial 
Participants 1682 middle school students, 31 teachers 
48% Male, 54% Latino 
Location California, United States 
Interventions Empathic Discipline is a brief intervention (i.e., online training, low cost) aimed at 


encouraging teachers to adopt an empathic mind-set about school discipline. The 
intervention encourages teachers to understand the values and perspectives of students 
which can cause misbehaviour. Teachers are empowered to handle difficult interactions, in 
a context of mutual understanding and trust. Empathic discipline is seen as an alternative to 
punitive, zero-tolerance practices. 


Programme deliverer 


Teachers 


Evaluator role 


Unclear 


Outcomes School suspension 
Respect to teachers 
Length One year 
Assessment Baseline and post treatment 
Attrition 13% 
COI statement Declared 
CoFl Unclear 
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27) Panayiotopoulos & Kerfoot (2004) 


Methods Randomised controlled trial 
Participants 124 secondary school students 
90% Male. Most of the students were White British. No percentage reported. 
Location Manchester, United Kingdom 
Interventions Home and School Support Project (HASSP) is delivered by a multidisciplinary team 


Programme deliverer 


offering rapid assessment and a treatment plan for the child, family and school staff. 
Treatment involves family therapy, individual CBT therapy and school support. 


Social worker, psychologist, community psychiatric nurse, play therapist 


Evaluator role Independent 
Outcomes School exclusion 
Disruptive antisocial behaviour and emotional symptoms 
Length Unclear 
Assessment Baseline, three months post-treatment 
Attrition 6% 
COl statement Not declared 


CoFl Unlikely. Intervention was delivered by community services (p.110) 
28) Peck (2006) 
Methods Randomised controlled trial 
Participants 1050 fifth through eighth grade school students 
59% Male, 62.5% Hispanic 
Location United States 
Interventions Student Targeted with Opportunities for Prevention (STOP program). It is a crime 


prevention programme offering a tutorial component, family and individual counselling, gang 
education as well as a drug and alcohol counselling intervention. The aim of the programme 
is that students will enter high school having what it takes to be a successful student and 
community member. 


Programme deliverers Probation Officer, community agencies 


Evaluator role Independent 


Outcomes School grades 
School attendance 
School suspension 
School expulsion 
Alcohol and drug use 


Contacts with the Juvenile Justice System 


Length One year 


Assessment Follow-up 


Attrition 39% 


COl statement Not presented 


CoFl Unlikely. Programme was implemented by the probation service in collaboration with public 
agencies (p. 41-3) 
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29) Reese, Murphy, & Filipezak (1981) 


Methods Randomised controlled trial 
Participants 98 seventh through ninth grade school students 
Gender is not reported, 100% Black 
Location United States 
Interventions Preparation through Responsive Education Programs (PREP). The intervention is designed 


Program deliverer 


to improve academic performance and social skills. Intervention involves reinforcement, 
and teaching adaptive behaviour as well as skills for self-control. 


Teachers 


Evaluator role 


Unclear 


Outcomes School grades 
School attendance 
School suspension 
School behaviour 
Length 27 weeks 
Assessment Baseline, during treatment 
Attrition Unknown 


COl statement 


Not presented 


CoFl Unlikely 
30) Russell (2007) 
Methods Randomised controlled trial 
Participants 61 sixth grade school students 
63.5% Male, 83% Caucasian 
Location Oregon, US 
Interventions Abbreviated version of Coping Power (CP) attempts to improve a child’s social competence, 


self-regulation, self-control and social bonds with peers, teachers and caregivers. CP 
incorporates individual counselling sessions, weekly group meetings and monthly parent 
meetings. 


Programme deliverer 


Psychologists, master level clinicians 


Evaluator role 


Independent evaluator 


Outcomes Adaptive functioning 
School referrals 
Detentions 
Suspensions 
Length 24 weeks 
Assessment Post treatment 
Attrition 14% 


COI statement 


Not presented 


CoFl 


Unlikely. Researcher is not programme developer, there is no evidence that the programme 
is commercialised. 
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31) Shetgiri, Kataoka, Lin, & Flores (2011) 


Methods Randomised controlled trial 
Participants 108 ninth grade school students 
Experimental group: 51% Male, 81% Latino 
Control group: 33% Male, 75% Latino 
Location California, United States 
Interventions School-based programme to reduce violence and substance use. Originally designed for 


White and African-American youths, the intervention is group based and aimed at 
increasing resilience through skills enhancement sessions (€.g., anger management, 
conflict resolution) and counselling sessions. 


Programme deliverer 


School clinical social worker 


Evaluator role Independent 
Outcomes Smoking 
Alcohol and drug use 
Fighting 
Grades 
Suspension 
Expulsion 
Length 28 weeks 
Assessment Baseline and eight month follow-up 
Attrition 20% 
COl statement Not presented 
CoFl Unlikely 
32) Smith (2004) 
Methods Randomised controlled trial 
Participants 40 eleventh to twelfth grade school students (Hispanic-serving schools) 
60% Male, 65% Hispanic-Latino 
Location Texas, United States 
Interventions The Personal Responsibility Group consists of instructional activity sessions on Emotional 


Intelligence skills (e.g., assertiveness, comfort and rapport, empathy, time-management, 
anxiety reduction, and motivation). The aim of the intervention is to assist students in 
identifying El strength and areas of growth and positive change. 


Programme deliverer 


Unknown 


Evaluator role 


Delivered the programme 


Outcomes 


Personal responsibility 

Constructive thinking 

Expectative 

Grades 

School behaviour 

In-school and out-of-school suspension 


Length 


10 weeks 


Assessment 


Baseline and post treatment 


Attrition 


No attrition 


COl statement 


Not presented 


CoFl 


Possible. Researcher delivered the intervention (p. 285) 
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33) Snyder et al. (2010) 
Methods 


Cluster randomised controlled trial 


Participants 544 students nested in 20 elementary schools 
Gender is not reported, 30% Part Hawai'i an 
Location Three Hawaiian Islands, US. 
Interventions Positive Action programme includes a K-12 classroom curriculum involving six components: 


self-concept, social and emotional positive actions for managing oneself responsibly, 
positive actions directed toward physical and mental health, honesty, getting along with 
others, and continually improving oneself. The programme also includes teacher, 
counsellor, family, and community training as well as activities directed toward school-wide 
climate development. 


Programme Deliverer 


Teachers, school administrators, school staff, counsellors 


Evaluator role 


Designed the programme 


Outcomes Absenteeism 
Suspension 
Retention in grades 
School achievements 
Length 35 weeks 
Assessment Baseline, post treatment, one year follow-up 
Attrition Unclear 
COl statement Declared 


CoFl 


Likely. “The research described herein was done using the program and the training and 
technical support of Positive Action, Inc. Dr. Flay’s spouse holds a significant financial 
interest in Positive Action, Inc.” (p.50) 
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34) Sprague, Biglan, Rusby, Gau, & Vincent (2016) 


Methods Cluster randomised controlled trial 
Participants 13,498 nested in 35 middle schools 
No data on gender; =70% White 
Location Oregon, US. 
Interventions School-Wide Positive Behavioural Interventions and Supports (SWPBIS) refers to a 


programme that targets a systemic change process in a whole school or in a district. 
SWPBIS aims to reduce students’ behaviour problems by changing staff behaviours and 
developing systems and supports to meet children’s behavioural needs. 


Programme Deliverer School staff, teachers and administrators 
Evaluator role Independent 
Outcomes In-school suspension, out-of-school suspension and expulsion 


School level achievements 
Antisocial behaviour and aggression 
Safety 

Substance use 

Relationship with teacher 
Attachment to school rules 


Length One year 

Assessment Baseline, post treatment and 12-month follow-up 

Attrition Unclear 

COl statement Not declared 

CoFl Unlikely 

35) Tilghman, (1988) 

Methods Randomised controlled trial 

Participants 100 sixth to eighth grade school students 
Gender and ethnicity are not reported 

Location United States 

Interventions Counsellor Peers are “students who have been properly trained through a certified peer 


counselling programme coordinator to work with and listen their peers. Rather than being an 
“advice-giver” or “problem-solver’, peer counsellors are sensitive listeners trained in 
communication skills to help their peers through the process of decision making and self- 
exploration.” (p.10) 


Programme deliverer Trained students 
Evaluator role Evaluator trained the counsellor peers 
Outcomes Attitude towards school, themselves and peers 
Suspension 
Length Nine weeks 
Assessment Unclear 
Attrition 9% 
COl statement Not presented 
CoFl Possible. Researcher was involved in the selection and training of counsellors. It is 


regarded as a limitation in the study (p. 9). No evidence that the programme is 
commercialised. 
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36) Ward & Gersten, (2013) 


Methods Cluster randomised controlled trial 
Participants 32 elementary schools; 7500 students 
Gender is not reported. Ethnic minorities represent =80% 
Location United States 
Interventions Safe and Civil Schools (SCS) is a school-wide intervention promoting positive behaviour 


Programme deliverer 


and support. “Program deliverers receive training on how to implement improvements 
related to safety, behaviour and discipline”. (p.320) 


School administrator, three general education teachers, one special education teacher and 
one or two other staff. 


Evaluator role Independent 
Outcomes Suspension 
Achievements test scores 
Resiliency, protective factors, and risk behaviours 
Opinion on the level of implementation of the school-wide intervention 
Length One year 
Assessment Post treatment and one-year follow-up 
Attrition Unclear 


COI statement 


CoFl 


Not presented 


Unlikely. Programme was developed by Sprick et al.1992 for profit but evaluator seems to 
be independent. 


37) Wyman et al. (2010) 


Methods Cluster randomised controlled trial (randomised within classrooms) 
Participants 226 kindergarten to third grade school students 
54% Male, more than 60% Black 
Location Rochester City, US 
Interventions Rochester Resilience Program: “In 14 lessons with school-based mentors, children were 


taught a hierarchical set of skills: monitoring of emotions; self-control/reducing escalation of 
emotions; and maintaining control and regaining equilibrium. Mentors provided classroom 
reinforcement of skill use”. (p. 707) 


Program deliverer 


Resilience Mentors 


Evaluator role 


Design the programme 


Outcomes Classroom behaviour 
Social-emotional functioning 
Office disciplinary referrals 
Out-of-school suspension 

Length 14 weeks 

Assessment Baseline and post treatment 

Attrition 13% 


COI statement 


Not presented 


CoFl 


Possible. Main author developed the programme (p.709). No information about 
commercialisation of the programme. 
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9.2 RISK OF BIAS IN INCLUDED STUDIES BASED ON EPOC RISK OF BIAS TOOL (PER STUDY) 


Sequence Equivalence Equivalence Selective outcome 
Study generation Concealment (outcome) (Characteristics) | Incomplete data Blinding Contamination reporting 
Allen et al. 1997 HR UR HR HR LR HR HR LR 
Arter, 2005 UR UR UR HR UR UR HR LR 
Barnes et al. 2003 HR UR HR LR HR UR LR LR 
Berlanga, 2004 UR UR HR HR UR HR HR LR 
Bradshaw et al. 2012 LR LR LR LR LR UR LR LR 
Bragdon, 2010 UR UR HR HR UR UR HR LR 
Brett, 1993 UR LR UR HR HR UR HR LR 
Burcham, 2002 LR LR LR UR HR UR HR LR 
Cook et al. 2014 LR LR LR LR LR UR HR LR 
Collier, 2002 UR UR LR LR LR HR HR LR 
Cornell et al. 2012 UR UR LR LR LR UR LR HR 
Crowder, 2001 UR HR UR UR LR UR HR LR 
Dynarski et al. 2003 UR LR LR LR UR UR HR LR 
Edmunds et al. 2012 HR UR UR LR UR UR HR LR 
Farrell et al. 2001 LR UR HR LR LR LR HR LR 
Feindler et al. 1984 UR UR LR HR LR LR HR LR 
Harding, 2011 UR UR UR HR HR HR HR LR 
Hawkins et al. 1988 UR UR LR LR UR HR HR LR 
Hirsch et al. 2011 LR LR LR LR LR UR HR LR 
Hostetler & Fisher 1997 UR UR UR HR HR HR HR LR 
lalongo et al. 2001 UR UR LR LR UR HR HR LR 
Johnson, 1983 LR LR LR LR LR HR HR LR 
Lewis et al. 2013 LR LR UR LR LR HR LR LR 
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9.3 EXTRACTED DATA FOR EFFECT SIZE CALCULATIONS: PRIMARY OUTCOME 


Author Type of Sample Mean* School-based Universal/ % FSM Cluster?’ Extracted data for Effect size Measure of 
Study Age Programme Indicated2é effect size calculations calculation exclusion 
(SD) 
1) Allen et al. Journal 695 15.8 Teen outreach Unclear Unknown No Suspension Effect size was Presence/absence 
(1997) article (1.13) Baseline calculated as the 
T1: 58 cases (17%); N=342 difference between 
C1: 81 cases (23.8%); baseline and post 
N=353 treatment. We 
Post treatment corrected final 
(immediately after calculations by adding 
treatment) the value of pre/post 
T2: 42 cases (13%); N=324 correlation, assumed 
C2: 93 cases (28.7%); to be equal .75. See 
N=323 methods section for 
further details 
2) Arter, (2005) Journal 52 Secondary Positive Indicated 40%FSM No Suspensions SMD was calculated N° of days 
article school Alternative No baseline reported using equations 3 and 
Learning Post Treatment 4 in the methods 
Support (presumably after section. 
(PALS) treatment) 
T2: M=.675; SD=.194; 
N=23 
C2: M= 675; SD=.227: 
N=17 
3) Barnes et al. Journal 45 16 Stress Universal Unknown No Suspension The principal N° of days 
(2003) article (1.3) reduction Baseline investigator provided N 


T1: M=0.8 days; SD=1.8; 
N=23 
C1: M=0.0 days; SD=.0; 
N=18 


size for T2. SMD was 
calculated as the 
difference between 
time 1 and time 2, 
accounting for the 
covariation between 


35 When the mean age was not available in the original study, their grade in school has been reported. Their school grade gives the reader a general idea of the age of the 


students. 


36 Universal intervention strategies are those oriented to reach the entire population of students, without regard to individual risk factors. Indicated programmes were defined 


as those targeting students displaying behavioural problems, punished at school or presenting a specific risk to their educational development. 
37 Due to the nature of the settings (schools) some studies reported clustered data. We corrected SE errors when it was needed. See methods section for further details. 
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During intervention 
T2(during): M= 0.5 days; 
SD=1.2; N=23 
C2(during): M=1.2 days; 
SD=3.0 N=18 


pre- and post 
measures (equations 8 
and 9 in the methods 
section). 


4) Berlanga 
(2004) 
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PhD 
Thesis 


80 


Eighth 
grade 


Grades, 
Attendance 
and Behaviour 
(GAB) 


Indicated 
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Unknown 


No 


In School Suspension 
Baseline 

T1: M=1.15; SD=1.29; 
N=32 

C1: M=.61; SD=1.14; N=31 
Post Intervention 
(immediately after 
treatment) 

T2: M=1.12; SD=1.21; 
N=32 

C2: M=1.03; SD=1.88; 
N=31 

Suspension 

Baseline 

T1: M=.31; SD=.53; N=32 
C1: M=.35; SD=.83; N=31 
Post Intervention 
(immediately after 
treatment) 

T2: M=.34; SD=.90; N=32 
C2: M=.58; SD=1.11; N=31 


Removal/Expulsion 
Baseline 

T1: M=.06; SD=.24; N=32 
C1: M=.06; SD=.24; N=31 
Post Intervention 
(immediately after 
treatment) 

T2: M=.03; SD=.17; N=32 
C2: M=.16; SD=.45; N=31 


Effect size was N° of events 
calculated as the 
difference between 
baseline and post 
treatment. We 
corrected final 
calculations by adding 
the value of pre/post 
correlation, assumed 
to be equal. 75. See 
methods section for 
further details 


5) Bradshaw et Journal 12,334 Elementary  School-Wide Universal 49% Yes Out-of-school suspension The study reports N° of events 
al. (2012) article school Positive FSM Follow-up (four years) results using multi- 
Behavioural Student level level analysis. In this 
Interventions Control: 21 schools case, we have not 
and Support (N=5124) Treatment: 16 applied any correction 
(SWPBIS) schools (N=6614) OR=.73; — of standard errors. We 
95% Cl .59 and .91 assume that MLM 
accounted for clusters 
and subsequently 
corrected the bias (see 
p. e1140). See 
methods section for 
further details 
6) Bragdon PhD Thesis 68 Teach Team Indicated 49% No Suspension Effect size was N° of days 
(2010) Project FSM Baseline calculated as the 
T1:M=.06; SD=.34; N= 34 difference between 
C1:M=.96; SD=2.42:; N=34 baseline and post 
Post treatment (during) treatment. We 
T2:M=.03; SD=.17; N= 34 corrected final 
C2:M=1.10; SD=2.61; N=34 calculations by adding 
Follow-up (three months _ the value of pre/post 
later) correlation, assumed 
T3:M=.09; SD=.51; N=34 to be equal. 75. See 
C3:M=1.07; SD=3.19; N=34 methods section for 
further details. 
7) Brett (1993) PhD Thesis 126 12-14 Efficacy, DC Universal | Unknown Yes School Suspension Based on Hedges N° of events 
years Clusters: 3 control (n=66) (2007) and Spier et al. 
3 Experimental (n=60) (2013), effect sizes 
Post-treatment (1 Month) — were computed using 
T2: M=.53; SD= 1.02; N=40 — d72, assuming equal 
C2: M=.63; SD=1.21; N=57; cluster sample size, p 
=.05. See methods 
section for further 
details 
8) Burcham PhD Thesis 71 Middle Social Indicated 38% No In-School Suspension Effect size was N° of events 
(2002) school problem FSM -Baseline calculated as the 
solving skills T1:M=8.62; SD=6.44; N=37 difference between 
training C1:M=7.88; SD=4.47; N=32 _ baseline and post 
p=.58 treatment. We 
corrected final 
calculations by adding 
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-lmmediately after 
treatment 

T2: f=.18 ; p= 0.67; N=69 
-18 months after 
treatment 

T3: f=.04; p=0.84; N=38 


Out-of-school 
Suspension 

Baseline 

T1: M=3.22; SD=3.71; 
N=37 

C1: M=2.56; SD=3.40; 
N=32 p=.45 
Immediately after 
treatment 

T2: f=1.09; p=.30; N=69 
18 months after treatment 
T3: f= 1.83; p=.18; N=38 


the value of pre/post 
correlation, assumed 
to be equal. 75. See 
methods section for 

further details 


9) Collier (2002) 
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PhD Thesis 60 Pro-social Indicated 


skills training 


5-14 years 
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Unknown 


No 


School Suspension (+) 
Baseline 

T1: M=1.93; SD=.4498: 
N=26 

C1: M=1.86; SD=.5074; 
N=25 

Post treatment 
(presumably after 
treatment) 

T2:M=1.15; SD=.6748; 
N=26 

C2:M=2.16; SD=.3742: 
N=25 


Study was identified as 
an outlier value. It was 
winsorised as 
suggested by Wilson & 
Lipsey 2001. 

Effect size was 
calculated as the 
difference between 
baseline and post 
treatment. We 
corrected final 
calculation by adding 
the value of pre/post 
correlation, assumed 
to be equal .75. 


N° of days 


10) Cook et al. Technical 106 BAM (skills- Indicated 26% No Baseline datais reported Datawasenteredinto N° of events 
(2014) report training) and FSM but incomplete CMA by using the 
MATCH Out-of-school suspension option Log OR and its 
(tutoring) (ITT) b=-.642; SE=.501; SE. No further 
(unclear the number of corrections. 
months/weeks of post Even if the evaluation 
treatment measured) was testing two 
different interventions, 
data was reported in a 
composite measure. 
The original author 
took that option 
because they 
recognised 
contamination between 
groups (spillover). 
11) Cornell et al. Journal 201 Threat Indicated = Unknown Yes Long-term suspension No corrections N° of events 
(2012) article assessment Post treatment 
(presumably after 
treatment) 
T2: 25 (25%): N=100 
C2: 49 (49%): N=101 
12) Crowder PhD Thesis 109 Gang Unclear Unknown No Out of School SMD was calculated N° of events 
(2001) Resistance, Suspension using equations 3 and 
Education and No baseline measure 4 in the methods 
Training reported section. No further 
(GREAT) Post intervention corrections. 
(presumably after 
treatment) 
T2: M=.1329; SD=.4629; 
N=53 
C2: M=.1429; SD=.1610; 
N=56 
In School Suspension 
No baseline measure 
reported 
Post intervention 
(presumably immediately 
after treatment) 
T2: M=.3584; SD=.7464; 
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N=53 
C2: M=.4464; SD=.8464; 
N=56 


13) Dynarski et Technical 968 Elementary 21s Century Unclear Unknown No Post treatment 12 months Datawasenteredinto N° students 
al. (2003:2004) report school Community (2003) CMA by using a 2x2 
Learning T2: 7.1% (38) N=537 table. No further 
C2: 5.2% (16) N=317 corrections. 
Follow up 24 months 
T3: 60 (6.2%) N=537 
C3: 43 (4.4%) N=317 
14) Edmunds et Journal 1607 15.3 Early College Unclear 50.6% No? % Suspended at least The principal N° of events 
al. (2012) article High School once investigator provided 
Academic T2: 6.4%; (57) N=885 measures for effect 
skills C2: 13.3%; (86) N=644 size calculation (via 
enhancing mail communication). 
15) Farrell et al. Journal 626 11.7 Responding in Universal | Unknown Yes In-school suspension Although the study is N° of events 
(2001) article (0.6) Peaceful and Post intervention based on clustered 
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(immediately after 
treatment) 

OR=5.0 (95%CI 1.5; 17.1) 
6 months 

OR=1.4 (95% Cl .7; 2.8) 
12 months 

OR=1.4 (95%CI .6; 3.0) 
Out-of-School 
Suspension 
Post-intervention 
(immediately after 
treatment) 

OR=0.9 (95%CI .5; 1.8) 
6 months 

OR=1.1 (95% Cl .6; 2.0 
12 months 

OR=0.9 (95% Cl .6; 1.4) 


data, we have not 
applied any correction 
of standard errors. The 
author mentions the 
use of GEE to 
calculate robust 
estimates of standard 
errors (see Farrell et 
al., 2001, p. 455). 


See methods section 
for further details. 


16) Feindler et Journal 36 13.8 Anger control Indicated = Unknown No School Expulsion Effect size was N° of events 
al. (1984) article (.68) training Baseline calculated as the 
T1: M=1.45; SD=.71:N=18 difference between 
C1: M=1.40; SD=.44; N=18 _ baseline and post 
Five-weeks follow-up treatment. We 
T2: M=.77; SD=.29; N=18 corrected final 
C2: M=.1.2; SD=.46; N=18 —_ calculation by adding 
the value of pre/post 
correlation assumed to 
be equal .75. See 
methods section for 
further details. 
17) Harding PhD Thesis 48 Eighth Over to you Indicated = Unknown No Six-months follow-up Data was entered into _—_N° of events 
(2011) grade Fixed term exclusion CMA by using a 2x2 
T2: 5 N=20 table. No further 
C2: 5 N=23 corrections. 
Permanent exclusion 
T2: 1 N=20 
C2: 1 N=23 
18) Hawkins et Journal 160 Seventh Proactive Indicated = Unknown No Times Suspended SMD was calculated N° of events 
al. (1988) article grade Classroom Post-treatment using equations 3 and 
Management (presumably after 4 in the methods 
treatment) section. No further 
T2: M=.48; SD=1.3; N=67 corrections. 
C2: M=.89; SD=2.1; N=75 
19) Hirsch et al, Technical 535 15.9 After School Indicated 86% No School Suspension Effect size was N° of events 
(2011) report Matters Baseline calculated as the 
T1: M=1.27; SD=.63; difference between 
N=259 baseline and post 
C1: M=1.29: SD=.64; treatment. We 
N=178 corrected final 
Post treatment calculations by adding 
(immediately after the value of pre/post 
treatment) correlation, assumed 
T2: M=1.36; SD=.72; to be equal .75. See 
N=259 methods section for 
C2: M=1.40; SD=.75; further details. 
N=178 
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20) Hostetler & Journal 317 Third grade Project CARE Indicated = Unknown No Suspension Effect size was N° of events 
Fisher (1997) article (Skill for Baseline calculated as the 
parents and T1: M=0.13; SD=0.56; difference between 
children) N=151 baseline and post 
C1: M=0.07; SD=0.35; treatment. We 
N=140 corrected final 
Post treatment (a few calculations by adding 
months after treatment, no __ the value of pre/post 
clear specification) correlation, assumed 
T2: M=0.20; SD=0.53; to be equal .75. See 
N=155 methods section for 
C2: M=0.25; SD=0.89 further details. 
N=141 
One year follow-up 
T3: M=0.26; SD=0.80; 
N=90 
C3: M=0.15; SD=0.66, 
N=86 
Two year follow-up 
T4: M=0.27; SD=0.74; 
N=30 
C4: M=0.09; SD=0.29; 
N=34 
21) lalongo et al. Journal 678 6.20 Two Universal 62.3% Yes Suspension Since data was Presence/absence 
(2001) article (.34) interventions FSM Five year follow-up dichotomous and 
i) Classroom- Classroom-centred vs. nested in clusters, we 
centred (CC) control corrected standard 
ii) Family- OR=.73; (95%Cl=.56; .95) —_ errors of effect sizes. 
school ** treatment group less The design effect was 
partnership likely to be suspended corrected by using the 
(FSP) Family-school formula suggested by 
partnership Higgins & Green 
OR=.59 (95%CI .35; .97) (2011) expressed by 


170 


The Campbell Collaboration | www.campbellcollaboration.org 


Boys: OR=1.13 (95%CI .61; 
2.09) 

Girls: OR=.38 (95%CI .17; 
.86) ** treatment group less 
likely to be suspended 


the equation [1+(M-1) 
x1]. See methods 
section for further 
details. 


22) Johnson PhD Thesis 60 Seventh ATTEND Indicated = Unknown No Suspension Effect size was N° of events 
(1983) and eighth (Counselling Baseline calculated as the 
grade and T1: M=.76; SD=.85; N=30 difference between 
monitoring) C1: M=.83; SD=.87; N=30 baseline and post 
Post intervention (after treatment. We 
treatment) corrected final 
T2: M=.36; SD=.55; N=30 calculations by adding 
C2: M=1.5; SD=1.25 N=30 __ the value of pre/post 
correlation, assumed 
to be equal .75. 
23) Lewis et al. Journal 624 Elementary Positive Action Universal Grade3 Yes Suspension The principal N° of events 
(2013) article school 84% Baseline (2001) investigator provided 
FSM T1: M=40.95; SD=48.13; data for calculations. 
N=3648 Based on Hedges 
C1: M=65.25; SD=56.15; (2007) and Spier et al. 
N=3800 (2013), effect sizes 
Post treatment (2004) were computed using 
T2: M=55.17; SD=64.84; drz , assuming equal 
N=3407 cluster sample size, p 
C2: M=77.63; SD=66.8; =.05.. See methods 
N=3687 section for further 
Follow-up (2005) details. 
T3: M=68.08; SD=80.02; 
N=3367 
C3: M=88.96; SD=76.56; 
N=3539 
24) Mack (2001) — PhD Thesis 20 Fourth to ICAN Kids! Indicated 95% (for No Out-of-school Effect size was N° of events 
sixth grade Behavioural school, Suspension calculated as the 
group no stated Baseline difference between 
counselling for T1: M=1.5000; SD=.9718; baseline and post 
sample) N=10 C1: treatment. We 
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M=1.9000; SD=.8756; 
N=10 

3 weeks(during) 
Tduring: M=.8000; 
SD=.6325; N=10 
Cduring: M=.9000; 
SD=.7379; N=10 
Post-treatment 

T2: M=.3000; SD=.4830; 
N=10 


corrected final 
calculations by adding 
the value of pre/post 
correlation assumed to 
be equal .75. See 
methods section for 
further details. 


C2: M=.4000; SD=.5164; 
N=10 

+3 weeks 

T3: M=.0000; SD=.0000; 
N=10 

C3: M=1.0000; SD=.6667; 
N=10 


25) Obsuth et al. Journal 738 13.9 Engage in Indicated 32% Yes Exclusion The study reports Presence/absence 
(2016) article Education Official Records results using multi- 
(Skills training) Baseline level analysis. In this 
T1: OR=2.784; SE=.300; case, we have not 
p=.001 applied any correction 
Post-treatment (1 month) of standard errors. We 
T2: OR=1.444; SE=.389; assume that MLM 
p=.344 accounted for clusters 
and subsequently 
corrected bias (see p. 
11). The study offered 
measures of impact 
based on self- 
reporting, teachers’ 
reports and official 
records. We extracted 
from OR based on 
official records (most 
of our studies report 
official records of 
suspension). 
26) Okonofua et Journal 1682 Middle Empathic Universal | Unknown Yes Suspension Since data was Presence/absence 
al. (2016) article school Discipline Post treatment (unclear dichotomous and 
number of months/weeks) __ nested in clusters, we 
T2: OR=.42; Z= -3.33; corrected standard 
p=.001; N=1449 errors of effect sizes. 
31 clusters The design effect was 
corrected by using the 
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formula suggested by 
Higgins & Green 
(2011) expressed by 
the equation [1+(M-1) 
x1]. See methods 
section for further 
details. Published data 
did not provide 
confidence intervals or 
SE. We tried to contact 
authors but it was not 
possible. We calculate 
an approximate 
SE=.013. 


27) Journal 124 10 Home and Indicated = Unknown No Exclusion Effect size was N° days 
Panayiotopoulos article School T1: M=9.50; SD=14.81; calculated as the 
& Kerfoot (2004) Support N=62 difference between 
Project C1: M=5.11; SD=7.56; N= baseline and post 
(HASSP) 62 treatment. We 
Post treatment (After corrected final 
three months) calculations by adding 
T2: M=4.95; SD=13.11; the value of pre/post 
N=61 correlation, assumed 
C2: M=5.51; SD=11.94; to be equal .75. 
N=62 
28) Peck (2006) = PhD Thesis 1050 Fifth to Student Unclear Unknown No Suspension Data was entered into N° of events 
eighth Targeted with Post-treatment (unclear CMA by using a 2x2 
grade Opportunities number of weeks/months __ table. No further 
for Prevention after treatment) corrections. 
(STOP) T2: 22; N=315 
C2: 22; N=321 
29) Reese et al. Journal 98 Seventh to Preparation Indicated © Unknown Matched Suspension Data was entered into N° of days 
(1981) article ninth grade — through peers During school year CMA by using X? 
Responsive T2 vs C2: X2 (1)= 6.58, originally reported. No 
Education p<.02 further corrections. 
Programs 
(PREP) 
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30) Russell PhD Thesis 61 11.5 Coping Power _— Indicated? += Unknown No Suspension SMD was calculated N° of events 
(2007) (.46) (Skills training Post treatment using equations 3 and 
for reducing T2(School A): M=.15; 4 in the methods 
aggression) SD=.38; N=13; section. No further 
C2(School B): M=.31; corrections. 
SD=.60; N=16; 
T2(School B: M=.29; 
SD=.61; N=14 
C2(School B):M=.00; 
SD=.00;N=10 
31) Shetguiri et Journal 108 14 Violence and Indicated 100% No Suspended or Expelled Effect size was Presence/absence 
al. (2011) article drug use Baseline calculated as the 
reduction T1: 8 (21%) N=40 difference between 
C1: 10 (22%) N=46 baseline and post 
Eight months follow-up treatment. We 
T2: 6 (14%) N=40 corrected final 
C2: 4 (8%) N=46 calculations by adding 
the value of pre/post 
correlation, assumed 
to be equal .75. 
32) Smith (2004) PhD Thesis 40 The Personal Indicated = Unknown No In-school Suspension SMD was calculated N° of events 
Responsibility Post-treatment based on f-test. No 
Group T2: f=11.085; p greater further corrections. 
(Emotional than or equal to .002; 
Intelligence Out-of-school 
skills) Suspension 
Post treatment 
T2: f= 10.088; p greater 
than or equal to .003 
33) Snyder et al. Journal 544 Elementary Positive Action — Universal 55% Yes Suspension (% of The principal N° of students 
(2010) article school students suspended) investigator provided 


children 
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2002 

T1: M=1.12; SD=1.10; 
N=5000 

C1: M=.98; SD=1.11; 
N=5000 

2006 

T2: M=.67; SD=.64; 
N=5000 

C2: M=1.72; SD=1.55; 


sample sizes for 
calculations. Based on 
Hedges (2007) and 
Spier et al. (2013), 
effect sizes were 
computed using d72 , 
assuming equal cluster 
sample size. Since the 
report presented the 


N=5000 

2007 

T3: M=.84; SD=.61; 
N=5000 

C3: M=2.53; SD=2.80; 
N=5000 


value of p, we used 
this value in 
calculations. 


34) Sprague et 
al. (2016) 
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Unknown 


Yes 


Expulsion 

Baseline 

T1: M=.002; SD=.004; 
N=6492 

C1: M=.003;SD=.004; 
N=7006 

Post treatment 

T2: M=.002; SD=.004; 
N=6492 

C2: M=.003; SD=.005; 
N=7006 


Follow-up (1 year later) 


T3: M=.003; SD=.006; 
N=6492 
C3: M=.003; SD=.004; 
N=7006 


In School Suspension 


Baseline 

T1: M=.071; SD=.094; 
N=6492 

C1: M=.135 ;SD=.189; 
N=7006 

Post treatment 

T2: M=.064; SD=.087; 
N=6492 

C2: M=.097; SD=.133; 
N=7006 


Follow-up (1 year later) 


Principal investigator N° of events 
provided data for 
calculations. Based on 
Hedges (2007) and 
Spier et al. (2013), 
effect sizes were 
computed using d72 , 
assuming equal cluster 
sample size, p =.05. 
See methods section 
for further details. 


T3: M=.058; SD=.060; 
N=6492 
C3: M=.095; SD=.145; 
N=7006 


Out-of-School 
Suspension 
Baseline 

T1: M=.082; SD=.063; 
N=6492 

C1: M=.078; SD=.065; 
N=7006 

Post treatment 

T2: M=.076; SD=.077; 
N=6492 

C2: M=.061; SD=.042:; 
N=7006 

Follow-up (1 year later) 
T3: M=.073; SD=.064; 
N=6492 

C3: M=.075; SD=.051; 
N=7006 


35) Tilghman PhD Thesis 100 12.5 Counsellor Indicated? = Unknown No Suspension Data was entered into —_N° of students 
(1988) Peers T2: 11 (N=46) CMA by using a 2x2 
C2: 26 (N=45) table. No further 
Significance test corrections. 
36) Ward & Journal 33 Elementary Safe and Civil Universal 90% Yes Post treatment (end of Since data was nested _N° of days 
Gersten (2013) article schools __ school Schools intervention) in clusters, we 
= children OR=.83; SE=.05 corrected standard 
Follow-up (1 year later) errors of effect sizes. 
25,000 (cumulative impact) The design effect was 
students OR=.77; SE=.04 corrected by using the 
formula suggested by 
record Higgins & Green 
(2011) expressed by 
the equation [1+(M- 
1)x1]. 
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37) Wyman et al. Journal 226 K - 3rd Rochester Indicated 90% Yes Suspension events Since data was nested _N° of events 


(2010) article Resilience Post-treatment in clusters, we 
Programme (immediately after corrected standard 
intervention) errors of effect sizes. 
TC2: Exp b=-0.57; The design effect was 
corrected by using the 
SE=0.23; z=-2.48; formula suggested by 
9=0.013 Higgins & Green 


(2011), expressed by 


Controlling for suspension the equation [1+(M-1) 


11 

x1]. See methods 
59 classrooms, 4 students section for further 
per classroom 


details 


Abbreviations: T1(treatment group baseline measure); C1 (control group baseline measure); T2 (treatment group post treatment measure); C2 (control group 
post treatment measure); T3(treatment group follow up measure); C3(control group follow up measure); FSM (free school meals); M (mean); SD (standard 
deviation); N (sample size); OR (Odds ratio); 95% Cl (95% confidence interval); SE (standard error); f (ANCOVA coefficient); p (p-value); b (beta coefficient); 
X? (chi-squared). 
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Figures 
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11. Data and analyses 


11.1 SUB-GROUP ANALYSES 


Forest plot sub-group analysis: Predominant gender in school 


Group by Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Gomis Std diff Standard Lower Upper 

in means error Variance limit limit Z-Value p-Value 
Male Obsuth etal 2016 Suspension -0.362 0.141 0.020 -0639 -0.085 -2560 0.010 
Male Arter 2005 Suspension 0.000 0.320 0.102 -0627 0627 0.000 1.000 
Male Bragdon 2010 Suspension 0.058 0.176 0.031 -0.287 0.403 0329 0.742 
Male Harding 2011 Combined 0.091 0.631 0.398 -1146 1328 0144 0.886 
Male Russel 2007 Combined 0.154 0.400 0.160 -0631 0938 0384 0.701 
Male Cook etal 2014 Out of school 0.354 0276 0.076 -0.187 0.895 1.281 0.200 
Male Panayiotopoulos etal 20045uspension 0.418 0.126 0.016 0.170 0666 3305 0.001 
Male Cornell etal 2012 Suspension 0.583 0.168 0.028 0.253 0.913 3.464 0.001 
Male Barnes etal 2003 Suspension 0.913 0.226 0.051 0470 1.356 4.043 0.000 
Male Smith 2004 Combined 1.045 0.336 0.113 0.386 1.704 3.109 0.002 
Male Collier 2002 Suspension 1.099 0.192 0.037 0.722 1.476 5.713 0.000 
Male 0.407 0.159 0.025 0096 0719 2560 0.010 
Mixed Shetguiri etal 2011 Suspension/Expulsion -0.398 0.251 0.063 -0890 0094 -1586 0.113 
Mixed Mack 2001 Out of school -0.232 0318 0.101 -0855 0.391 -0.730 0.465 
Mixed Bradshaw etal 2012 Out of school -0.174 0.061 0.004 -0.293 -0.054 -2847 0.004 
Mixed Peck 2006 Suspension 0.011 0.172 0.030 -0.327 0349 0.065 0.948 
Mixed Hirsch etal 2011 Suspension 0.023 0.071 0.005 -0.116 0162 0325 0745 
Mixed Burcham 2002 Combined 0.049 0.170 0.029 -0.285 0382 0285 0776 
Mixed Crowder 2001 Combined 0.070 0.192 0.037 -0306 0445 0363 0716 
Mixed Brett 1993 Suspension 0.090 0.265 0.070 -0429 0609 0340 0.734 
Mixed lalongo etal 2001 SuspensionCC 0.169 0.094 0.009 -0.014 0.353 1.806 0.071 
Mixed Hostetler etal 1997 Suspension 0.196 0.084 0.007 0.032 0.360 2343 0.019 
Mixed Dinarsky etal 2003 Suspension 0.198 0.169 0.029 -0.133 0.530 1172 0.241 
Mixed Hawkins etal 1988 Suspension 0.232 0.169 0.028 -0.099 0.562 1375 0.169 
Mixed lalongo etal 2001b SuspensionFSM 0.289 0.176 0.031 -0.056 0635 1.641 0.101 
Mixed Berlanga 2004 Combined 0.317 0.179 0.032 -0.034 0.668 1.772 0.076 
Mixed Lewis etal 2013 Suspension 0.330 0.100 0.010 0134 0526 3.300 0.001 
Mixed Farrell etal 2001 Combined 0.415 0.276 0.076 -0.127 0.956 1500 0.134 
Mixed Edmunds etal 2012 Suspension 0.440 0.100 0.010 0.244 0636 4400 0.000 
Mixed Okonofua etal 2016 Suspension 0.478 0.014 0.000 0451 0.505 34680 0.000 
Mixed Wyman 2010 Suspension 0.659 0.408 0.166 -0.141 1458 1615 0.106 
Mixed 0.168 0.076 0.006 0019 0317 2216 0.027 
Overall 0.213 0.069 0.005 0.078 0.347 3.102 0.002 

-4.00 -2.00 4.00 
Favours Control Favours Treatment 
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Forest plot sub-group analysis: Grade at school 


Group by Study name Outcome Statistics for each study ‘Sid diff in means and 95% Cl 
Grade Std diff Standard Lower Upper 

inmeans error Variance limit limit Z-Value p-Value 
Elementary Mack 2001 Out of school 0.232 0318 0101 0855 0391 -0730 0.465 
Elementary Bradshaw etal 2012 Out of school 0.174 0.061 0.004 -0293 -0054 -2847 0.004 
Elementary Ward 2013 Suspension 0.100 0.087 0.007 -0069 0270 1.159 0.246 
Elementary lalongo etal 2001 SuspensionCC 0.169 0.094 0.009 -0014 0353 1.806 0.071 
Elementary Hostetler etal 1997 Suspension 0.196 0.084 0.007 0032 0360 2343 0.019 
Elementary Dinarsky etal 2003 Suspension 0.198 0.169 0029 -0133 0530 1172 0241 
Elementary lalongo etal 2001b SuspensionF SM 0.289 0.176 0.031 -0.056 0.635 1641 0.101 
Elementary Lewis etal 2013 Suspension 0.330 0.100 0010 0134 0526 3300 0.001 
Elementary Panayiotopoulos etal 2004Suspension 0.418 0126 0016 0170 0686 3305 0.001 
Elementary Snyder 2010 Suspension 0.610 0316 0100 -0010 1230 1.929 0.054 
Elementary Wyman 2010 Suspension 0.659 0.408 0166 0141 1458 1615 0.106 
Elementary Collier 2002 Suspension 1.099 0.192 0037 0722 1476 §713 0.000 
Elementary 0270 0.091 0008 0092 0447 2972 0.003 
High Shetguiri etal 2011 SuspensiowExpulsion -0.398 0.251 0.063 -0890 0094 -1586 0.113 
High Hirsch etal 2011 Suspension 0.023 0.071 0005 -0116 0162 0325 0.745 
High Allen etal 1997 Suspension 0.343 0077 0006 0191 0495 4428 0000 
High Cook etal 2014 Out of school 0.364 0.276 0076 -0.187 0.895 1.281 0.200 
High Edmunds etal 2012 Suspension 0.440 0.100 0010 0244 0636 4400 0.000 
High Barnes etal 2003 Suspension 0913 0.226 0051 0470 1356 4043 0.000 
High ‘Smith 2004 Combined 1.045 0.336 0113 0366 1704 3109 0.002 
High Feindler etal 1984 Expulsion 1,203 0.245 0060 0723 1683 4911 0.000 
High 0.447 0.138 0019 O178 O717 3.251 0.001 
Middle Obsuth etal 2016 Suspension 0.362 0141 0.020 -0639 -0085 -2560 0.010 
Middle Arter 2005 Suspension 0.000 0.320 0102 -0627 0627 0.000 1.000 
Middle Peck 2006 Suspension 0.011 0.172 0.030 -0327 0349 0.065 0.948 
Middle Burcham 2002 Combined 0.049 0.170 0.029 -0285 0382 0.285 0776 
Middle Bragdon 2010 Suspension 0.068 0.176 0031 -0287 0403 0329 0.742 
Middle Crowder 2001 Combined 0.070 0.192 0.037 -0306 0445 0363 O716 
Middle Sprague 2016 Combined 0.080 0.100 0.010 -0116 0276 0800 0.424 
Middle Brest 1993 Suspension 0.090 0.265 0070 -0429 0609 0340 0734 
Middle Harding 2011 Combined 0.091 0.631 0398 -1146 1328 0144 0.886 
Middle Russel 2007 Combined 0.154 0.400 0160 -0631 0936 0384 0701 
Middle Hawkins etal 1988 Suspension 0.232 0.169 0028 -0099 0562 1.375 0.169 
Middle Berlanga 2004 ‘Combined 0317 0.179 0.032 -0.034 0.668 1.772 0.076 
Middle Farrell etal 2001 Combined 0415 0276 0076 -0127 0956 1.500 0.134 
Middle Okonofua etal 2016 Suspension 0478 0.014 0.000 0451 0505 34680 0.000 
Middle Tiighman 1988 Suspension 0811 0.253 0064 0315 1307 3.206 0.001 
Middle Johnson 1983 Suspension 1,099 0.192 0037 0722 1476 §713 0.000 
Middle 0.227 0.093 0009 0044 0410 2431 0.015 
Overall 0.285 0.069 0003 0170 0400 4848 0.000 

4.00 -2.00 2.00 4.00 
Favours Control Favours Treatment 
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Forest plot sub-group analysis: Type of intervention 


Group by 


interv2 


Academic support 
Academic support 
Academic support 
After school 

After school 

After school 
Counselling 
‘Counselling 
‘Counselling 
‘Counselling 
Mentoring/monitoring 
Mentoring/monitoring 
Mentoring/monitoring 
Mentoring/monitoring 
Mentoring/monitoring 
Mentoring/monitoring 
Other 

Other 

Other 


School-wide 
Schoolwide 
School-wide 
‘School-wide 
‘School-wide 
School-wide 
School-wide 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for students 

ST for teachers 

ST for teachers 

ST for teachers 

ST for teachers 

ST for teachers 
Violence reduction 
Violence reduction 
Violence reduction 
Violence reduction 
‘Overall 
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Study name 


‘Cook ets! 2014 
Edmunds ets! 2012 


Hirsch etal 2011 
Dinarsky etal 2003 


Berlangs 2004 
Pansyiotopoulos ets! 2004 
Tilghman 1988 


Peck 2006 
Brett 1993 
Reese ets! 1981 
Wyman 2010 
Johnson 1983 


Arter 2005 
Bragdon 2010 
‘Crowder 2001 
Allen etal 1997 


Bradshaw etal 2012 
Sprague 2016 
Ward 2013 

Lewis etal 2013 
Comell ets! 2012 
Snyder 2010 


Shetguiri ete! 2011 
Obsuth ets! 2016 
Burcham 2002 
Harding 2011 
Russel 2007 
Hostetler etal 1997 
Barnes etal 2003 
Smith 2004 
Collier 2002 


Islongo etal 2001 
Hawkins etal 1988 
lalongo etal 2001b 
‘Okonofus ets! 2016 


Mack 2001 
Ferrell etal 2001 
Feindler etal 1984 


Qutcome 


Out of school 
Suspension 


Suspension 
Suspension 


Combined 
Suspension 
Suspension 


Suspension 
Suspension 
Suspension 
Suspension 
Suspension 


Suspension 
Suspension 
Combined 

Suspension 


Out of school 
Combined 
Suspension 
Suspension 
Suspension 
Suspension 


Suspension/Expulsion 
Suspension 
Combined 

Combined 

Combined 
Suspension 
Suspension 
Combined 


Suspension 


SuspensionCC 
Suspension 
SuspensionFSM 
Suspension 


Out of school 
Combined 
Expulsion 
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Std diff 
in means 


0.354 
0.440 
0.420 
0.023 
0.198 
0.049 
0.317 
0.418 
0.811 
0.457 
0.011 
0.090 
0.537 
0.659 
1.099 
0.472 
0,000 
0.058 
0.070 
0.343 
0.207 
0.174 
0.080 
0.100 
0.330 
0.583 
0.610 
0.198 
0.398 
0.262 
0.049 
0.091 
0.156 
0.196 
0.913 
1.045 
1.099 
0.207 
0.169 
0.232 
0.289 
0.478 
0.313 
0.232 
0.415 
1.203 
0.481 
0.240 


Standard 
error 


0.276 
0.100 
0.094 
0.071 
0.169 
0.065 
0.179 
0.126 
0.253 
0.116 
0.172 
0.265 
0.212 
0.408 
0.192 
0.232 
0.320 
0.176 
0.192 
0.077 
0.093 
0.061 
0.100 
0.087 
0.100 
0.168 
0.316 
0.116 
0.251 
0.141 
0.170 
0.631 
0.400 
0.086 
0.226 
0.326 
0.192 
0.184 
0.094 
0.169 
0.176 
0.018 
0.104 
0.318 
0.276 
0.245 
0.415 
0.036 


Variance 


0.076 
0.010 
0.009 
0.005 
0.029 
0.004 
0.032 
0.016 
0.064 
0.013 
0.030 
0.070 
0.045 
0.166 
0.037 
0.054 
0.102 
0.031 
0.037 
0.006 
0.008 
0.004 
0.010 
0.007 
0.010 
0.028 
0.100 
0.016 
0.063 
0.020 
0.029 
0.398 
0.160 
0.007 
0.051 
0.113 
0.037 
0.034 
0.009 
0.028 
0.031 
0.000 
0.011 
0.101 
0.076 
0.060 
0.173 
0.001 


limit 
0.187 
0.246 
0.246 
0.116 
0.133 
0.079 
0.034 
0.170 
0.315 
0.230 
0.327 
0.429 
0.120 
0.141 
0.722 
0.017 
0.627 
0.287 
0.206 
0.191 
0.026 
0.293 
0.116 
0.069 
0.134 
0.253 
0.010 
0.020 
0.890 
0.639 
0.285 
1.146 
0.631 
0.032 
0.470 
0.386 
0.722 
0.054 
0.014 
0.099 
0.056 
0.451 
0.108 
0.855 
0.127 
0.723 
0.334 
0.169 


Statistics for each study 


Lower Upper 


limit 


0.895 
0.636 
0.614 
0.162 
0.530 
0.177 
0.668 
0.666 
1.307 
0.683 
0.249 
0.609 
0.953 
1.458 
1.476 
0.927 
0.627 
0.403 
0.445 
0.495 
0.389 
0.054 
0.276 
0.270 
0.526 
0.913 
1.230 
0.426 
0.094 
0.085 
0.382 
1.328 
0.938 
0.260 
1.356 
1.704 
1.476 
0.668 
0.353 
0.562 
0.635 
0.505 
0.517 
0.391 
0.956 
1.683 
1.295 
0.312 


ZValue  p-Value 


1.281 
4.400 
4.573 
0.325 
1.172 
0.752 
1.772 
3.205 
3.206 
3.952 
0.065 
0.340 
2.528 
1.615 
5.713 
2.034 
0.000 
0.329 
0.263 
4.428 
2.242 
-2.847 
0.800 
1.159 
3.200 
3.464 
1.928 
1.698 
“1.586 
-2.560 
0.285 
0.144 
0.384 
2.343 
4.043 
3.109 
5.713 
1.665 
1.806 
1.375 
1.641 
34.680 
2.999 
0.730 
1.500 
4911 
1.187 
6.624 


0.200 
9.000 
0.000 
0.745 
0.241 
0.452 
0.076 
0.001 
0.001 
0.000 
0.948 
0.734 
0.012 
0.106 
0.000 
0.042 
1,000 
0.742 
0.716 
0.000 
0.025 
0.004 
0.426 
0.246 
0.001 
0.001 
0.056 
0.089 
0.113 
0.010 
0.776 
0.886 
0.701 
0.019 
0.000 
0.002 
0.000 
0.096 
0.071 
0.169 
0.101 
0.000 
0.003 
0.465 
0.134 
0.000 
0.247 
0.000 


-4.00 


Std diff in means and 95% Cl 


Favours Control 


2.00 4.00 


Favours Treatment 


Forest plot sub-group analysis: Theoretical bases of interventions 


Group by __ Study name Qutcome Statistics for each study Std diff in means and 95% CI 
ss Std diff Standard Lower Upper 
in means error Variance limit limit Z-Value p-Value 
Individual Shetguiri etal 2011 Suspension/Expulsion -0.398 0.251 0.063 -0890 0.094 -1.586 0.113 
Individual Obsuth etal 2016 Suspension -0.362 0.141 0.020 -0639 -0.085  -2.560 0.010 
Individual Mack 2001 Out of school 0.232 0.318 0101 -0855 0391 -0.730 0.465 
Individual Arter 2005 Suspension 0.000 0.320 0.102 -0627 0627 0.000 1.000 
Individual Peck 2006 Suspension 0.011 0.172 0.030 -0.327 0.349 0.065 0.948 
Individual Hirsch etal 2011 Suspension 0.023 0.071 0.005 -0.116 0.162 0.325 0.745 
Individual Burcham 2002 Combined 0.049 0.170 0.029 -0.285 0382 0.285 0.776 
Individual Bragdon 2010 Suspension 0.058 0.176 0.031 -0.287 0403 0.329 0.742 
Individual Crowder 2001 Combined 0.070 0.192 0.037 -0306 0445 0.363 0.716 
Individual Brett 1993 Suspension 0.090 0.265 0.070 -0429 0609 0.340 0.734 
Individual Harding 2011 Combined 0.091 0.631 0.398 -1146 1328 0.144 0.886 
Individual Russel 2007 Combined 0.154 0.400 0.160 -0631 0938 0.384 0.701 
Individual Hostetler etal 1997 Suspension 0.196 0.084 0.007 0.032 0.360 2.343 0.019 
Individual Dinarsky etal 2003 Suspension 0.198 0.169 0.029 -0.133 0.530 4.172 0.241 
Individual Berlanga 2004 Combined 0.317 0.179 0.032 -0.034 0.668 1.772 0.076 
Individual Allen etal 1997 Suspension 0.343 0.077 0.006 0.191 0.495 4.428 0.000 
Individual Cook etal 2014 Out of school 0.354 0.276 0.076 -0.187 0.895 1.281 0.200 
Individual Farrell etal 2001 Combined 0.415 0.276 0.076 -0.127 0.956 1.500 0.134 
Individual Panayiotopoulos etal 2004Suspension 0.418 0.126 0.016 0.170 0.666 3.305 0.001 
Individual Edmunds etal 2012 Suspension 0.440 0.100 0.010 0244 0636 4.400 0.000 
Individual Reese etal 1981 Suspension 0.537 0.212 0.045 0.120 0.953 2.526 0.012 
Individual Wyman 2010 Suspension 0.659 0.408 0166 -0.141 1.458 1.615 0.106 
Individual Tilghman 1988 Suspension 0.811 0.253 0.064 0315 1.307 3.206 0.001 
Individual Barnes etal 2003 Suspension 0.913 0.226 0.051 0470 1.356 4.043 0.000 
Individual Smith 2004 Combined 1.045 0.336 0.113 0386 1.704 3.109 0.002 
Individual Collier 2002 Suspension 1.099 0.192 0.037 0.722 1.476 5.713 0.000 
Individual Johnson 1983 Suspension 1.099 0.192 0.037 0722 1476 5713 0.000 
Individual Feindler etal 1984 Expulsion 1.203 0.245 0.060 0723 1683 4911 0.000 
Individual 0.334 0.072 0.005 0193 0476 4633 0.000 
School Bradshaw etal 2012 Out of school -0.174 0.061 0.004 -0.293 -0.054 -2847 0.004 
School Sprague 2016 Combined 0.080 0.100 0.010 -0.116 0276 0.800 0.424 
School Ward 2013 Suspension 0.100 0.087 0.007 -0.069 0.270 1.159 0.246 
School lalongo etal 2001 SuspensionCC 0.169 0.094 0.009 -0.014 0.353 1.806 0.071 
School Hawkins etal 1988 Suspension 0.232 0.169 0.028 -0.099 0.562 1.375 0.169 
School lalongo etal 2001b SuspensionFSM 0.289 0.176 0.031 -0.056 0.635 1.641 0.101 
School Lewis etal 2013 Suspension 0.330 0.100 0.010 0134 0.526 3.300 0.001 
School Okonofua etal 2016 Suspension 0.478 0.014 0.000 0451 0.505 34.680 0.000 
School Cornell etal 2012 Suspension 0.583 0.168 0.028 0253 0.913 3.464 0.001 
School Snyder 2010 Suspension 0.610 0.316 0.100 -0.010 1.230 1.929 0.054 
School 0.246 0.105 0.011 0041 0452 2.352 0.019 
Overall 0.306 0.059 0.004 0.189 0.422 5.150 0.000 
4.00 -2.00 4.00 
Favours Control Favours Treatment 
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Forest plot sub-group analysis: Training before intervention 


Group by Study name 
Training 

Unreported Mack 2001 
Unreported Peck 2006 
Unreported Burcham 2002 
Unreported Bragdon 2010 
Unreported Crowder 2001 
Unreported Berlanga 2004 
Unreported Cook etal 2014 
Unreported Panayiotopoulos etal 2004 
Unreported Edmunds etal 2012 
Unreported Reese etal 1981 
Unreported Smith 2004 
Unreported Collier 2002 
Unreported 

Yes Shetguiri etal 2011 
Yes Obsuth etal 2016 
Yes Bradshaw etal 2012 
Yes Arter 2005 

Yes Hirsch etal 2011 
Yes Sprague 2016 

Yes Brett 1993 

Yes Harding 2011 

Yes Ward 2013 

Yes Russel 2007 

Yes lalongo etal 2001 
Yes Hostetler etal 1997 
Yes Dinarsky etal 2003 
Yes Hawkins etal 1988 
Yes lalongo etal 2001b 
Yes Lewis etal 2013 
Yes Allen etal 1997 
Yes Farrell etal 2001 
Yes Okonofua etal 2016 
Yes Cornell etal 2012 
Yes Snyder 2010 

Yes Wyman 2010 

Yes Tilghman 1988 
Yes Bames etal 2003 
Yes Johnson 1983 

Yes Feindler etal 1984 
Yes 

Overall 
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Outcome 


Out of school 
Suspension 
Combined 
Suspension 
Combined 
Combined 
Out of school 
Suspension 
Suspension 
Suspension 
Combined 
Suspension 


Suspension/Expulsion 
Suspension 
Out of school 
Suspension 
Suspension 
Combined 
Suspension 
Combined 
Suspension 
Combined 
SuspensionCC 
Suspension 
Suspension 
Suspension 
SuspensionFSM 
Suspension 
Suspension 
Combined 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Expulsion 


Std diff Standard 


in means 


-0.232 
0.011 
0.049 
0.058 
0.070 
0.317 
0.354 
0.418 
0.440 
0.537 
1.045 
1.099 
0.338 

-0.398 

-0.362 

-0.174 
0.000 
0.023 
0.080 
0.090 
0.091 
0.100 
0.154 
0.169 
0.196 
0.198 
0.232 
0.289 
0.330 
0.343 
0.415 
0.478 
0.583 
0.610 
0.659 
0.811 
0.913 
1.099 
1.203 
0.291 
0.307 


error 


0.318 
0.172 
0.170 
0.176 
0.192 
0.179 
0.276 
0.126 
0.100 
0.212 
0.336 
0.192 
0.097 
0.251 
0.144 
0.061 
0.320 
0.071 
0.100 
0.265 
0.631 
0.087 
0.400 
0.094 
0.084 
0.169 
0.169 
0.176 
0.100 
0.077 
0.276 
0.014 
0.168 
0.316 
0.408 
0.253 
0.226 
0.192 
0.245 
0.069 
0.056 


Statistics for each stud 


Variance 


0.101 
0.030 
0.029 
0.031 
0.037 
0.032 
0.076 
0.016 
0.010 
0.045 
0.113 
0.037 
0.009 
0.063 
0.020 
0.004 
0.102 
0.005 
0.010 
0.070 
0.398 
0.007 
0.160 
0.009 
0.007 
0.029 
0.028 
0.031 
0.010 
0.006 
0.076 
0.000 
0.028 
0.100 
0.166 
0.064 
0.051 
0.037 
0.060 
0.005 
0.003 


Lower 


limit 


0.855 
0.327 
0.285 
0.287 
0.306 
-0.034 
0.187 
0.170 
0.244 
0.120 
0.386 
0.722 
0.148 
-0.890 
-0.639 
-0.293 
0.627 
0.116 
0.116 
0.429 
1.146 
0.069 
-0.631 
-0.014 
0.032 
0.133 
0.099 
0.056 
0.134 
0.191 
0.127 
0.451 
0.253 
0.010 
0.141 
0.315 
0.470 
0.722 
0.723 
0.155 
0.196 


Upper 
limit 
0.391 
0.349 
0.382 
0.403 
0.445 
0.668 
0.895 
0.666 
0.636 
0.953 
1.704 
1.476 
0.528 
0.094 

0.085 
0.054 
0.627 
0.162 
0.276 
0.609 
1.328 
0.270 
0.938 
0.353 
0.360 
0.530 
0.562 
0.635 
0.526 
0.495 
0.956 
0.505 
0.913 
1.230 
1.458 
1.307 
1.356 
1.476 
1.683 
0.426 
0.417 


ZNValue p-Value 


-0.730 
0.065 
0.285 
0.329 
0.363 
1.772 
1.281 
3.305 
4400 
2.526 
3.109 
5.713 
3.491 
1.586 
-2.560 
-2.847 
0.000 
0.325 
0.800 
0.340 
0.144 
1.159 
0.384 
1.806 
2.343 
1.172 
1.375 
1.641 
3.300 
4.428 
1.500 
34.680 
3.464 
1.929 
1.615 
3.206 
4.043 
5.713 
4.911 
4.201 
5.447 


0.465 
0.948 
0.776 
0.742 
0.716 
0.076 
0.200 
0.001 
0.000 
0.012 
0.002 
0.000 
0.000 
0.113 
0.010 
0.004 
1.000 
0.745 
0.424 
0.734 
0.886 
0.246 
0.701 
0.071 
0.019 
0.244 
0.169 
0.101 
0.001 
0.000 
0.134 
0.000 
0.001 
0.054 
0.106 
0.001 
0.000 
0.000 
0.000 
0.000 
0.000 


4.00 


Favours Control 


Std diffin means and 95% Cl 


-2.00 
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4.00 


Favours Treatment 


Forest plot sub-group analysis: Monitoring the implementation of the 


intervention 


Group by 
Monitoring 


Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Unreported 
Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 
Overall 


Study name 


Obsuth ets! 2016 
Mack 2001 

Pec 2006 

Bragdon 2010 
Crowder 2001 

Brett 1993 

Harding 2011 
Russel 2007 
Hostetler ets! 1997 
Hawkins etal 1988 
Berlsngs 2004 
Allen etal 1997 
Cook ets! 2014 
Pansyiotopoulos etal 2004 
Edmunds etal 2012 
Okonofua etal 2016 
Reese ets! 1981 
Wyman 2010 
Tilghman 1988 
Bares ets! 2003 
‘Smith 2004 

Collier 2002 
Feindler ets! 1984 


Shetguiri etal 2011 
Bradshaw etal 2012 
Aster 2005 

Hirsch etal 2011 
Burcham 2002 
Sprague 2016 
Ward 2013 

lalongo etal 2001 
Dinarsky etal 2003 
lalongo etal 20016 
Lewis etal 2013 
Farrell etal 2001 
Comell ets! 2012 
Snyder 2010 
Johnson 1983 


Outcome 


Suspension 
Out of school 
Suspension 
Suspension 
Combined 
Suspension 
Combined 
Combined 
Suspension 
Suspension 
Combined 
Suspension 
Out of schoo! 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Suspension 
Combined 
Suspension 
Expulsion 


Suspension/Expulsion 
‘Out of school 
Suspension 
Suspension 
Combined 
Combined 
Suspension 
SuspensionCC 
Suspension 
SuspensionFSM 
Suspension 
Combined 
Suspension 
Suspension 


Suspension 


Std iff 
in means 
0.62 
0.232 
0.011 
0.058 
0.070 
0.080 
0.091 
0.184 
0.196 
0.232 
0.317 
0.343 
0.354 
0.418 
0.440 
0.478 
0.537 
0.668 
ott 
0.913 
1.045 
1.088 
1,203 
0.371 
0.398 
0.174 
0.000 
0.023 
0.049 
0.080 
0.100 
0.169 
0.198 
0.289 
0.330 
0.415 
0.583 
0.610 
1.099 
0.203 
0.301 


Standard 
error 
0.141 
0.318 
0.172 
0.176 
0.192 
0.265 
0.631 
0.400 
0.084 
0.169 
0.179 
0.077 
0.276 
0.126 
0.100 
0.014 
0.212 
0.408 
0.253 
0.226 
0.336 
0.192 
0.245 
0.064 
0.251 
0.061 
0.320 
0.071 
0.170 
0.100 
0.087 
0.094 
0.169 
0.176 
0.100 
0.276 
0.168 
0.316 
0.192 
0.076 
0.049 


Statistics for each study 
Lower Upper 
Variance limit —_ limit 
0.020 0.639 = -0.085 
0.101 0.855 0,391 
0.020 0.327 0.249 
0.021 0.287 0.403 
0.037 0.206 © 0.445 
0.070 0.429 0,609 
0.398 1.146 © 1.228 
0.160 0.631 0.938 
0.007 0.032 0.360 
0.028 0.099 0.862 
0.032 0.034 0,668 
0.006 0.191 0.495 
0.076 0.187 0.895 
0.016 0.170 0.cee 
0.010 0.244 0.636 
0.000 0.451 0.805 
0.045 0.120 0,953 
0.166 0.141 1.468 
0.064 60.315 = 1.207 
0.081 0.470 «1.266 
0.113 0.386 © 1.704 
0.027 0.722 «1.476 
0.060 0.723 «1.683 
0.004 0.246 «(0.498 
0.063 0.890 0.094 
0.004 0.293 0.054 
0102 0.627 0.627 
0.005 0.116 0.162 
0.029 0.285 0.382 
0.010 0.116 0.276 
0.007 0.069 0.270 
0.009 0.014 0.262 
0.029 0.133 0.630 
0.031 0.056 0.635 
0.010 0.134 «0.626 
0.076 0.127 0.986 
0.028 0.253 «0.913 
0.100 9.010 1,230 
0037 0.722 «(1.476 
0.006 0.054 0.281 
0,002 0.206 0.397 


ZValue 
-2,660 
0.730 
0.065 
0.328 
0.263 
0.240 
0,144 
0.284 
2.343 
1.37% 
1.772 
4.428 
1,281 
2.205 
4.400 
24.680 
2.526 
1.615 
2.206 
4.043 
3.109 
5.713 
4311 
5816 
“1,586 
2847 
0.000 
0.325 
0.285 
0.800 
1.159 
1.806 
1172 
1.641 
3.300 
1.500 
2.404 
1.928 
5713 
2676 
6.173 


pValue 
0.010 
0.465 
0.948 
0.742 
0.716 
0.734 
0.886 
0.701 
0.019 
0.168 
0.076 
0.000 
0.200 
0.001 
0.000 
0.000 
0.012 
0.106 
0.001 
0.000 
0.002 
0.000 
0.000 
0.000 
0.113 
0.004 
1.000 
0.745 
0.776 
0.424 
0.246 
0.071 
0.241 
0.101 
0.001 
0.134 
0.001 
0.054 
0.000 
0.007 
0.000 


and 95% Cl 


o 
¢ 


-4.00 2.00 0.00 2.00 4.00 


Favours Control Favours Treatment 
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Forest plot sub-group analysis: Reasons for conducting the research 


Group by Study name Qutcome Statistics for each study Std diff in means and 95% Cl 
= Std diff Standard Lower Upper 

inmeans error Variance limit limit Z-Value p-Value 
Demonstration Mack 2001 Out of school -0.232 0.318 0.101 -0.855 0.391 -0.730 0.465 
Demonstration Arter 2005 Suspension 0.000 0.320 0.102 -0627 0627 0.000 1.000 
Demonstration Hirsch etal 2011. Suspension 0.023 0.071 0.005 -0.116 0162 0325 0.745 
Demonstration Harding 2011 Combined 0.091 0.631 0.398 -1146 1.328 0.144 0.886 
Demonstration Russel 2007 Combined 0.154 0.400 0.160 -0631 0938 0384 0.701 
Demonstration _lalongo etal 2001 SuspensionCC 0.169 0.094 0.009 -0.014 0353 1806 0.071 
Demonstration _lalongo etal 2001b SuspensionFSM 0.289 0.176 0.031 -0.056 0.635 1.641 0.101 
Demonstration —_ Lewis etal 2013 Suspension 0.330 0.100 0.010 0.134 0526 3300 0.001 
Demonstration Cook etal 2014 Out of school 0.354 0.276 0.076 -0.187 0895 1281 0.200 
Demonstration  Okonofua etal 2016 Suspension 0.478 0.014 0.000 0451 0505 34680 0.000 
Demonstration Cornell etal 2012 Suspension 0.583 0.168 0.028 0.253 0913 3464 0.001 
Demonstration Snyder 2010 Suspension 0.610 0.316 0.100 -0.010 1.230 1.929 0.054 
Demonstration = Wyman 2010 Suspension 0.659 0.408 0.166 -0.141 1458 1615 0.106 
Demonstration Smith 2004 Combined 1.045 0.336 0.113 0386 1.704 3109 0.002 
Demonstration Collier 2002 Suspension 1.099 0.192 0.037 0722 1476 5713 0.000 
Demonstration Johnson 1983 Suspension 1.099 0.192 0.037 0722 1476 5713 0.000 
Demonstration 0.428 0.084 0.007 0264 0593 5105 0.000 
Routine Shetguiri etal 2011 Suspension/Expulsion -0.398 0.251 0.063 -0890 0.094 -1586 0.113 
Routine Obsuth etal 2016 Suspension -0.362 0.141 0.020 -0639 -0.085 -2560 0.010 
Routine Bradshaw etal 2012Out of school -0.174 0.061 0.004 -0.293 -0.054 -2847 0.004 
Routine Peck 2006 Suspension 0.011 0.172 0.030 -0.327 0.349 0.065 0.948 
Routine Burcham 2002 Combined 0.049 0170 0.029 -0.285 0382 0285 0776 
Routine Bragdon 2010 Suspension 0.058 0.176 0.031 -0.287 0403 0.329 0.742 
Routine Crowder 2001 Combined 0.070 0.192 0.037 -0.306 0445 0363 0716 
Routine Sprague 2016 Combined 0.080 0.100 0.010 -0116 0276 0800 0.424 
Routine Brett 1993 Suspension 0.090 0.265 0.070 -0429 0609 0340 0734 
Routine Ward 2013 Suspension 0.100 0.087 0.007 -0.069 0.270 1.159 0.246 
Routine Hostetler etal 1997 Suspension 0.196 0.084 0.007 0.032 0360 2343 0.019 
Routine Dinarsky etal 2003 Suspension 0.198 0.169 0.029 -0.133 0530 1.172 0.241 
Routine Hawkins etal 1988 Suspension 0.232 0.169 0.028 -0.099 0562 1375 0.169 
Routine Berlanga 2004 Combined 0.317 0.179 0.032 -0.034 0668 1772 0.076 
Routine Allen etal 1997 Suspension 0.343 0.077 0.006 0191 0495 4428 0.000 
Routine Farrell etal 2001 = Combined 0.415 0.276 0.076 -0.127 0956 1500 0134 
Routine Edmunds etal 2012 Suspension 0.440 0.100 0.010 0244 0636 4400 0.000 
Routine Tilghman 1988 Suspension 0.811 0.253 0.064 0315 1307 3206 0.001 
Routine 0.128 0.063 0.004 0004 0252 2028 0.043 
Overall 0.237 0.051 0.003 0.138 0336 4693 0.000 

4.00 -2.00 2.00 4.00 
Favours Control Favours Treatment 
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Forest plot sub-group analysis: Evaluator role 


Group by Study name Outcome Statistics for each study Std diff in means and 95% Cl 
Ewe? Std diff Standard Lower Upper 

inmeans error Variance limit limit Z-Value p-Value 
Dependent Mack 2001 Out of school -0.232 0.318 0.101 -0855 0391 -0730 0.465 
Dependent Arter 2005 Suspension 0.000 0.320 0.102 -0627 0627 0.000 1.000 
Dependent Burcham 2002 Combined 0.049 0.170 0.029 -0.285 0.382 0.285 0.776 
Dependent Harding 2011 Combined 0.091 0.631 0.398 -1146 1.328 0144 0.886 
Dependent lalongo etal 2001 SuspensionCC 0.169 0.094 0.009 -0.014 0353 1806 0.071 
Dependent lalongo etal 2001b SuspensionFSM 0.289 0.176 0.031 -0.056 0635 1641 0.101 
Dependent Lewis etal 2013 Suspension 0.330 0.100 0.010 0.134 0.526 3300 0.001 
Dependent Farrell etal 2001 Combined 0.415 0.276 0.076 -0.127 0.956 1.500 0.134 
Dependent Okonofua etal 2016 Suspension 0.478 0.014 0.000 0451 0505 34680 0.000 
Dependent Cornell etal 2012 Suspension 0.583 0.168 0.028 0.253 0.913 3464 0.001 
Dependent Snyder 2010 Suspension 0.610 0.316 0.100 -0.010 1.230 1929 0.054 
Dependent Wyman 2010 Suspension 0.659 0.408 0.166 -0.141 1458 1615 0.106 
Dependent Tilghman 1988 Suspension 0.811 0.253 0.064 0.315 1.307 3.206 0.001 
Dependent Smith 2004 Combined 1.045 0.336 0.113 0.386 1.704 3.109 0.002 
Dependent Collier 2002 Suspension 1.099 0.192 0.037 0.722 1476 5713 0.000 
Dependent Johnson 1983 Suspension 1.099 0.192 0.037 0.722 1476 5713 0.000 
Dependent 0.471 0.076 0.006 0.322 0.621 6.189 0.000 
Independent Shetguiri etal 2011 Suspension/Expulsion -0.398 0.251 0.063 -0.890 0.094 -1586 0.113 
Independent Obsuth etal 2016 Suspension -0.362 0.141 0.020 -0639 -0.085 -2560 0.010 
Independent Bradshaw etal 2012 Out of school -0.174 0.061 0.004 -0.293 -0.054 -2847 0.004 
Independent Peck 2006 Suspension 0.011 0.172 0.030 -0.327 0349 0.065 0.948 
Independent Hirsch etal 2011 Suspension 0.023 0.071 0.005 -0.116 0162 0325 0.745 
Independent Bragdon 2010 Suspension 0.058 0.176 0.031 -0.287 0403 0.329 0.742 
Independent Crowder 2001 Combined 0.070 0.192 0.037 -0.306 0445 0.363 0.716 
Independent Sprague 2016 Combined 0.080 0.100 0.010 -0.116 0276 0800 0.424 
Independent Brett 1993 Suspension 0.090 0.265 0.070 -0429 0609 0340 0.734 
Independent Ward 2013 Suspension 0.100 0.087 0.007 -0.069 0.270 1.159 0.246 
Independent Russel 2007 Combined 0.154 0.400 0.160 -0.631 0.938 0.384 0.701 
Independent Hostetler etal 1997 Suspension 0.196 0.084 0.007 0.032 0360 2343 0.019 
Independent Dinarsky etal 2003 Suspension 0.198 0.169 0.029 -0133 0530 1172 0.241 
Independent Hawkins etal 1988 Suspension 0.232 0.169 0.028 -0.099 0.562 1.375 0.169 
Independent Berlanga 2004 Combined 0.317 0.179 0.032 -0.034 0668 1.772 0.076 
Independent Panayiotopoulos etal 200Suspension 0.418 0.126 0.016 0170 0666 3305 0.001 
Independent Edmunds etal 2012 Suspension 0.440 0.100 0.010 0244 0636 4400 0.000 
Independent Barnes etal 2003 Suspension 0.913 0.226 0.051 0470 1.356 4.043 0.000 
Independent 0.125 0.062 0.004 0.003 0247 2004 0.045 
Overall 0.264 0.048 0.002 0.170 0359 5474 0.000 

~4.00 -2.00 4.00 
Favours Control Favours Treatment 
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Forest plot sub-group analysis: Type of exclusion 


‘Group by Study name Outcome Statistics for each study Std ditt im means and 95% Cl 
Typeexds Stddiff Standard Lower Upper 

inmeans error = Varlance «limit «= imit | Z-Value p-Value 
-* Marding 20115 Expulsion o082 ors O638 -1484 1646 0.101 0919 
on Sprague 20160 Expulsion 0230 o100 O010 0034 0436 2.300 oo21 
mn Gerlanga 20068 Expulsion os8s 0179 0032 O034 OF36 2.182 0032 
ct Feindler etal 1984 Expulsion 1203 o2s D050 0723 1683 4ai1 0.000 
ot os27 o233 2054 0070 OSES 2.261 o024 
a ‘Shetguiri etal 2041 Suspension/Expulsion D398 oO251 0063 0890 OO94 “1.586 O13 
e Obsuth etal 2016 Suspension 0.362 0141 G020 -0699 -O08S -2560 0.010 
z Russel 20070 Suspension OS1L 0376 O141 -1648 0425 0.829 0407 
e Arter 2005 Suspension 0.000 0320 0.102 -0627 0627 0000 1000 
2 Peck 2006 Suspension oon 0172 G030 0327 0349 o065 o948 
ft Hirsch etal 2021 Suspension 0.023 0071 6.005 0.116 0162 03525 0.745 
z Sragdan 2020 Suspension 058 0.176 «= .032 0.287 0403 0529 O72 
t Brett 1993 Suspension 0.090 0265 0.070 0429 0609 0.540 0734 
Ps Ward 2013 Suspension 0.100 cosy 0.007 Coes o270 1189 0206 
© Harding 2011 Suspension 0.101 0398 «0.159 0.680 0881 (0.252 0.801 
z lalomgo etal 20028 SuspensionOC o169 oose 0200S -00lé 0353 1.806 oor. 
@ Hostetler etal 1997 Suspension o196 ooss 0007 0032 O360 2.343 oo19 
z Dinarsky etal 2003 Suspension D196 o.169 m025 -0.133 O530 1.172 o241 
@ Hawkins etal 1983 Suspension 0232 o169 0028 0099 OS62 1375 o169 
t lalongo etal 20016 SuspensionFSM 0.289 0.176 «©6051 0.056 0635 1.641 0.101 
2 Lewis etal 2013 Suspension 0330 0.100 0010 0134 0526 3.300 0001 
e Allen ets! 1997 Suspension O43 0077 = .006 «0.191 0495 4.428 0.000 
Py Panayistopoulos etal 2004 Suspension Osis 0126 0016 O170 O866 3305 D001 
t Edmunds ete! 2012 Suspension 0440 0100 © 0.010 0.244 «0636 © 4.400 = 0.000 
= ‘Okonofua etal 2016 Suspension os7s ool ©000 0451 O505 36.689 0.000 
e Reese etal 1981 Suspension 0537 0212 6.065 0.120 0953 2526 0.012 
z Cornell etal 2012 Suspension 0583 0168 0028 0253 O913 3.468 0001 
Ff Smyder 2010 Suspension 0.610 0316 0200 0010 1290 1929 O54 
z Russel 2007b Suspension_ o6le o424 O179 -O212 Léeag 1460 ola 
2 Wyman 2010 Suspension 659 0408 O166 C141 1458 1615 0.106 
z Tilghman 158 Suspension Os1L o253 0064 0315 1307 3.206 ‘0.001 
@ Barnes etal 2003 Suspersion 0913 0226 O051 O470 1356 4043 0.000 
z Collier 2002 Suspension Loss O.192 0037 O722 L476 5.713 ‘0.000 
z Johnsen 1983 Suspension 1.099 0192 0037 O722 1476 5713 0.000 
c 0313 0056 «© .003 0.203 «0423 «5574 0.000 
is Burcham 20028 in school 0.030 0170 «= 02-0304 0564 0176 0860 
is Crowder 20018 inschoal 0.110 0192 0.087 0266 0486 (0574 0566 
is Sprague 2016b In schoal 0.280 0.100 «6G01D O08 0476 2800 0.005 
is Berlangs 200¢b in schoal 0.386 0.179.052 0.035 OF3T «62.158 0.032 
is Farrell etal 20022 in schoal ga? 0342 O.117 O216 1558 2552 0010 
is Smith 20045 In schoal 1.050 0337 «= @.lld 0389 «1711 «3.111 0.002 
is o245 0.120 2014 a110 OSEO 2.877 0.004 
05 Sprague 2016¢ Out of schoo! 270 0100 «G01 0465 -0074 -2,700 «0.007 
os Mack 2001 Out of school “D232 o31s O10 855 O39] “0.720 O465 
05 Bradshaw etal 2012 ‘Out of schoo! “0.174 O06] O004 0.295 -0054 -2847 0.004 
os Farrell etal 2001b Out of school -D.O5e O.189 0036 0428 O312 0.308 O75 
os Crowder 2001 Out of schoo! 089 0192 «0037 -0346 0405 O52 0879 
os Gurcham 20026 Gut of school 0.067 0170 «028-0267 0401 0595 O64 
8 Berlanga 2004¢ ‘Out of school 0.180 0179 0032 -O171 0531 1006 0314 
os Cook etal 2024 Out of school 0354 0276 «O76 0.187 0895 1281 0.200 
as Smith 20045 Out of schoo! 1.040 0335 O112 O38 1696 5408 0.002 
os ool 0092 «6.008 0.163 0199 O191 oBas 
Overall 025 oos¢ 0.002 O.173 0344 53907 D000 

-8.00 4.00 4.00 
Favours Treatment Favours Control 
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12. Data collection instruments 


12.1 ELIGIBILITY CHECK LIST 


Criteria 


1. Does this paper measure school exclusion as an outcome? 

2. Does the intervention is school based? (or at least one 
component in the school) 

3. Are the target individuals school students? 

4. The report is based in an experimental, quasi-experimental 
design? 

5. Is this report informing statistical results able to be transformed 
in effect sizes? 

6. Is this report included? 


Evaluation 


© Yes 
© Yes 


© Yes 
© Yes 


© Yes 


© Yes 


No 
No 


No 
No 


No 


No 


Reasons for exclusion: 
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12.2 DATA COLLECTION INSTRUMENT 


DATA-CODING INSTRUMENT 
School-based interventions for reducing 
disciplinary school exclusion. A systematic review 
[Variable names in brackets] 


Contents 


Section A. Codification 

Section B. Bibliographical information 
Section C. Ethics 

Section D. Research design 

Section E. Sample 

Section F. Primary outcome coding 
Section G. Secondary outcomes coding 
Section H. Base-line measurements 
Section |. Programme delivered 
Section J. Follow-up measurement 
Section K. Effect sizes 


Section A. Codification 

Instruction: use one data-coding instrument for each manuscript. When more than one manuscript reports the same research 
project, select one of them as the principal (e.g., the older) and give it an ID number. The following manuscripts should use 
the same ID but it must be registered in the Crossref field. 


[STUDYID] Study ID number: 

[CROSSREF'1] Cross reference document identifier: 
[CROSSREF2] Cross reference document identifier: 
[CROSSREF3] Cross reference document identifier: 
[DATESCR] Date of screening: 

[CODER] Coder Initials: 


Section B. Bibliographical information 


Before completing this section, please be sure that the manuscript is correctly uploaded in the reference manager 
programme. 


[AUTHOR] Name of the main author(s): 
[AFFIL] Main author affiliation: 
[DATEPUB] Year of publication: 


[DATEFIEDW] Year of fieldwork (usually reported in a range): 
[COISTATEMENT] Has the paper included a conflict of interest statement? 
© 1. Yes 
© 0.No 
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[LANGPUB] Language of the publication: 


MG) 


COROROMS) 


1. English 

2. German 

3. Italian 

4. Spanish 

5. Portuguese 

999. Other. 


[COUNTPUB] Country of publication: 


[TYPUB] Type 


2 O©GOOOO 


O©OOOOGOO 


fp 


1. UK 

2. USA 

3. Canada 

4. Australia 

999. Other. 
99. Unknown 


ublication: 

1. Journal 

2. Book/book chapter 

3. Masters thesis 

4. PhD/doctoral thesis 

5. Technical/governmental report 
6. Conference proceedings 

999. Other: 


[AUTDIS] Main author discipline: 


[LOCAT] How 


Section C. Ethics 


= ©O©OOOOOO 


CRORORORORC) 


1. Education 

2. Social Work 

3. Psychology 

4. Criminal Justice 

5. Sociology 

6. Psychiatry/Medicine 
999. Other: = 
99. Unknown 


as the study/report located? 


1. Electronic database 

2. Web search 

3. Reference in a book/paper. Please specify: 
4. Hand search in specialised journal 

5. Peer/expert suggestion 

999. Other. Specify: 


[CONSENT] Did the study declare the use of “consent agreement forms”? 


© 
© 
© 
© 


1. Yes 

0. No 

999. Other 
99. Unknown 


[SIGNCONS] Who signed the consent? 
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O®O©OOOOO 


1. Students 

2. Parents 

3. Teachers 

4. Schools 

5. Parents and student 
999. Other. Specify: 
99. Unknown 
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Section D. Design 


The present systematic review includes randomised control trials as well as quasi-experimental reports (before/after 
measure plus a control or comparison group). If the control/comparison group is randomly allocated, non-randomly allocated 
or matched and no intervention expected to produce impact is provided to it, you will be able to code that group as 
CONTROL. Subsequently, the TREATMENT group could be understood as the group that receives the intervention, no 
matter if that condition has been randomly allocated or not. 

Please select always the data that is related with the sample effectively analysed. 


[DESTYPE] What kind of design is this paper based on? 
1. Randomised controlled trial (true experiment) 
2. Before-and-after with control/comparison group/s 
3. Instrumental variable 
4. Propensity score matching 
5. Interrupted time series 
6. Pre/post measures with unmatched control/comparison group 
7. Inverse probability weighting 
999. Other. Specify:__ 
[RANDUNIT] Units of randomization 
© 1. Individuals 
© 2. Clusters/groups (classroom, schools) 
© 999. Other. Specify: 
© 99. Unknown 
[ANALUNIT] Unit of analysis 
© 1. Students 
© 2. Clusters/groups (classroom, schools) 
© 999. Other. Specify: 
© 99. Unknown 


[COMPVAR] Variables measured to create comparability? (e.g., variables used to match the control and treatment groups) 


(o) 


© 
© 
© 
© 
© 
© 


© 


[MAINSTAT] What is the main statistical analysis used to produce the final results? 
© 1. Multilevel modelling 
© 2. Differences of means 
© 3.MANOVA 
© 4. Chi-squared 
5. Propensity Score Matching 
999. Other. Specify: 


MG) 


Section E. Sample 


[SAMPSELECT] How was the sample selected? 
© 1. Randomly 
© 2. Assessment 
© 3. Self-selection 
© 999. Other. Specify: 


[INSAMP] Initial sample size (i.e., individuals/schools): 
[NUMBFOLL] N° of follow-up: 

[FOLLSAMP'1] Follow-up 1 sample size: 
[FOLLSAMP2] Follow-up 2 sample size: 
[FOLLSAMP3] Follow-up 3 sample size: 


[NSCHOOL] Initial number of schools: 
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[NSFOLL1] Follow-up 1 sample size: 
[NSFOLL2] Follow-up 2 sample size: 
[NSFOLL3] Follow-up 3 sample size: 


[NCLASS] Initial number of classes: 

[NCFOLL1] Follow-up 1 sample size: 
[NCFOLL2] Follow-up 2 sample size: 
[NCFOLL3] Follow-up 3 sample size: 


Please code here the information on attrition described in the manuscript: 


Total number Total number of 
of students at students at Follow-up 
Baseline 
Treatment [NTREBAS] [NTREFOLL] 
Control [NCONTBA] [NCONTFOL] 
[MEANAGE] Mean age and standard deviation of overall sample at beginning of intervention: 


[GENDER] Gender 
@© % of males 
© % of females 
@® 99. Unknown 
[LOCAT] Location of programme 

© Urban area 

© Suburban area 

© Rural area 

© Mixture of areas 

© 99. Not enough information to determine 


[GRADEX] Grade level of students 
© % of students in Elementary school or equivalent 
© % of students in Secondary school or equivalent 
© % of students in High school or equivalent 
© 4. Other: 
® 99. Unknown 


[ETHNI] Predominant ethnicity3® 

1. % of Caucasian: 

. % of Black: 

3. % of Hispanic: 

4. % of Asian: 

5. % of other mixed background: 

99. Unknown 

[COUNTRY] Please state the name of the country where schools and sample of students were located when tested. 
(99 if unknown) 


[LUNEX] Socio-economic status 


CRORCRORORS) 
i) 


% of students receiving free/reduced school lunch: 
99. Unknown 

[SENEX] Special Educational Needs 

% of students declaring SEN: 

99. Unknown 


38 Based on Lipsey & Wilson (2001) 
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Section F. Primary Outcome (School Exclusion) 


[EXCLUSION] Is the manuscript reporting outcomes for school exclusion? 
© 1. Yes 
© 0.No 


[TYPEXC] Type of exclusion measured 
© 1. In-school exclusion 
© 2. Out-of-school exclusion 
© 99. Unknown 


[CHEKTIP] Duration of school exclusion measured 
© 1. Days of Fixed-term exclusion 
(Expressed in number or days, frequencies, percentages) 
© 2. Days of Permanent exclusion 
(Expressed in number or days, frequencies, percentages) 
© 99. Unknown 


[ICCEXCLU] If the statistical analysis include cluster in MLM, please register the ICC for Exclusion: 


Section G. Secondary outcomes 


[BEHAVMES] Did the study include measures on behaviour domains? 
© 1. Yes 
© 0.No 
© 99. Unknown 


What types of the following behaviours are measured? 


e [PROSO] Pro-social behaviour (e.g., helping, empathy). Specify: 
© 1. Yes 
© 0.No 


[MEPROSO] Measure(s) used to test the behaviour (name): 
[ALPHAPROSO] 

© Reliability test. Specify alpha value: 

© Non reported 


Groups Effect size before Effect size after 
Control or comparison [PROBC] [PROAC] 
Treatment [PROBT] [PROAT] 


[PAGEPROSO] Number of the page from where you extract statistical data: 
[ICCPROSO] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: 


e [INTERNAL] Internalising problem behaviour 
(e.g., anxiety, depression, attention-deficit and hyperactivity disorder (ADHD), attention deficit, hyperactivity). 


Specify: 
© 1. Yes 
© 0.No 


[MINTERNAL] Measure (s) used to test the behaviour (name): 
[ALPHAINTERNAL] 

© Reliability test. Specify alpha value: 

© Non reported 
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Groups Effect size before Effect size after 


Control or comparison [PROBC] [PROAC] 
Treatment [PROBT] [PROAT] 


[PAGEINTERNAL] Number of the page from where you extract statistical data: 


[ICCINTERNAL] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: 


e [NAEXTERNAL] Non-aggressive externalising problem behaviour 


(e.g., stealing, lying, graffiti, illegal drugs). Specify: 
© 1. Yes 
© 0.No 


[MNAEXTERNAL] Measure used to test the behaviour (name): 
[ALPHANAEXTER] 

© Reliability test. Specify alpha value: 

© Non reported 


Groups Effect size before Effect size after 
Control or comparison [PROBC] [PROAC] 
Treatment [PROBT] [PROAT] 


[PAGENAXTERN] Number of the page from where you extract statistical data: 


[ICCNAEXT] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes: 


e [AAGRESEXT] Aggressive externalising problem behaviour 
(e.g., Opposition/defiance, physical aggression, indirect aggression, instrumental aggressions/dominance, reactive 


aggression, school bullying). Specify: 
© 1. Yes 
© 0.No 


[MAGRESSEXT] Measure used to test the behaviour (name): 
[ALPHAAEXT] 

© Reliability test. Specify alpha value: 

© Non reported 


Groups Effect size before Effect size after 
Control or comparison [PROBC] [PROAC] 
Treatment [PROBT] [PROAT] 


[AGRESPAGE] Number of the page from where you extract statistical data: 


[ICCAGREEX] If the statistical analysis include cluster in MLM, please register the ICC for Behavioural outcomes: 


Section H. Base-line measurements 


[DATABAS] Date of baseline assessment: 
What measures were used? 


[SRMES] Self-report 
© 1. Yes 
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@© 0.No 

@© 99. Unknown 
[TRMES] Teachers’ report 

© 1. Yes 

© 0.No 

© 99. Unknown 
[SCHRMES] School records 

© 1. Yes 

© 0.No 

© 99. Unknown 
[PAREP] Parents 

© 1.Yes 

© 0.No 

@® 99. Unknown 
[OMES] Other: 


[EXCBL] Frequency of exclusion at baseline (register any measure given by the study) 


Section |. Programme delivered 


This section aims to codify data on the delivery process. Be aware that sometimes final reports do not describe all the data 
related to delivery. In those cases it would be helpful to search for registered protocols or earlier publications reporting more 
data on this. 

[PRONAME] Name of the programme: 


[PROCURRI] Was the programme curricular? 
© 1. Yes 
© 0.No 
© 99. Unknown 
© 999. Other. Specify: 


[PROEND] The programme was conducted for: 
1. Research ends 

2. Demonstration ends 

3. Routine 

99. Unknown 

® 999. Other. Specify: 


[PROSIT] Primary programme site: 
© 1. Public school 
© 2. Private school 
© 3. Other, (specify): 
© 99. Unknown 


[PROSCH] Was at least one of the components of the intervention was settled at school? 
© 1.Yes 
© 0.No 
© 99. Unknown 


[PRODEL] Who delivered the programme? 
© 1. External facilitators 
© 2. School facilitators 
© 3.Both 
® 99. Unknown 


[PDBACK] Deliverer’s background 1 
© 1. Social worker 

2. Psychologist 

3. Teacher 

4. Police officers 

5. Peers 


© 
© 
© 
© 


© 
© 
© 
© 


195 The Campbell Collaboration | www.campbellcollaboration.org 


® 999. Other. Specify: 
® 99. Unknown 


[PDBACK] Deliverer’s background 2 
1. Social worker 
© 2. Psychologist 

© 3. Teacher 

© 4. Police officers 
© 

© 


) 


5. Peers 
999. Other. Specify: 
99. Unknown 


[TRAINBEF] Did the deliverer receive training BEFORE implementing the programme? 
© 1. Yes. 
© 0.No. 
© 99. Unknown 

[THOURS] How long was the training in hours?: 


[TRAINDUR] Did the deliverer receive training DURING the implementation? 
© 1. Yes. 
© 0.No. 
© 99. Unknown 


[THOURS2] How long was the training in hours?: 


o) 


What type of intervention was delivered? If the manuscript indicates a mixture of interventions you can 
than one using TYPEPRO 1, 2 and 3. 


select more 


[TYPEPRO1] [TYPEPRO2] 


[TYPEPRO3}] 


1. Mentoring programme 


2. Restorative programme 


3. Skills training programme 


4. School-wide systemic intervention 


5. Classroom management 


6. Counselling/therapy 


999. Other 


Theoretical background of the intervention. If the manuscript indicates a mixture of theories, you can select more than one 


using THEORY 1, 2 and 3. 


[THEORY1] [THEORY2] 


[THEORY3] 


1. Cognitive behavioural 


2. Learning theory 


3. Restorative theories 


4. Organisational theories or principles 


99. Unknown 


999. Other (Specify) 


[PROCONT] What happened to the control group? 
© 1.No intervention 
© 2. Wait-list control 
© 3. Minimal contact 
© 4. Treatment as usual 
© 5. Alternative treatment 
© _ 5. Placebo 
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© 999. Other. Specify: 


[PROFORN] Delivery format: 

1. Manualised programme 
2. Unstructured programme 
3. Mixed 

99. Unknown 

999. Other. Specify: 


What was the programme dosage? 
[PRODOSW] AVERAGE Duration in weeks: 
[PRODOSH] AVERAGE Hours per week: 


[PROFREQ] What was the frequency of the programme counted? 
1. Less than a week 

2. Once a week 

3. Twice a week 

4. 3-4 times a week 

5. Daily 

99. Unknown 


was the “evaluator” role? 
1. Deliver the programme 
2. Designed the programme 
3. Both design and delivery 
4. Independent evaluator 
99. Unknown 
[MONITOR] Was the programme implementation monitored? 
© 1.Yes 
© 0.No 
© 99. Unknown. Not enough information 


CRORCROMS) 


2 ©8©0008 


[EVROLE] What 


CROROROMS) 


[IMPROB] Does the report provide information about implementation problems? 
© 1. Yes, there were clear problems which are reported 
© 0.No, non-reported problems, reasonably well implemented 
© 2. Possible problems based on the description of the intervention 
© 99. Unknown. Not enough information 


[PROCOST] Is the cost of the intervention mentioned? 
© 1. Yes 
© 0.No 

[AMOUNT] Cost: 

[UNITCURR] Currency: 


Section J. Follow-up measurement 


[DATEFALL] Date of follow up: 

Multiple follow-ups 

[MONTHFO1] N° of months from baseline to 1 follow-up: 
[MONTHFO2] N° of months from baseline to 2" follow-up: 
[MONTHFO3] N° of months from baseline to 3” follow-up: 
[MONTHF04] N° of months from baseline to 4" follow-up: 
What measures were used? 


[POSTSR] Children/adolescent self-report 
© 1. Yes 
© 0.No 


[POSTTR] Teachers’ report 
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© 1.Yes 


@© 0.No 
[POSTSR] School records 

© 1. Yes 

© 0.No 
[POSTPR] Parents report 

© 1. Yes 

@® 0.No 
[POSTO] Other: 


[FREQEXFOLL] Frequency of exclusion at follow-up (register any measure given by the study) 


Section K. Effect sizes of intervention on school exclusion 


e Effect size: outcomes expressed in continuous data. 
[CSSEX] Sample size for the ES (Treatment group) 
[CSSCON] Sample size for the ES (Control group) 


[MEANEX] Mean (Treatment group) 
[MEANCON] Mean (Control group) 


[MEANADJ] Are the Means adjusted? 
© 1. Yes. 
@® 0.No 


[ADJBY] Adjusted by (describe): 


[SDEX] Standard deviation (Treatment group) 
[SDCON] Standard deviation (Control group) 


[SEEX] Standard error (Treatment group) 
[SECON] Standard error (Control group) 


[CORREX] Correlation coefficient + p value (Treatment group) 
[CORRCON] Correlation coefficient + p value (Control group) 


[SMDTREAT] Standardised mean difference + confidence intervals 


e Effect size: outcomes expressed in dichotomous data. 


[DSSTRE] Sample size for the ES (Treatment group) 
[DSSCONT] Sample size for the ES (Control group) 


[NUMTRE] Treatment group; number of successful cases: 
[NUMCON] Control group; number of successful cases: 


[PROPTRE] Treatment group; proportion of successful cases: 
[PROPCON] Control group; proportion of successful cases: 
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[ORTRE] Treatment group; odds ratios: 
Confidence Intervals: 
p-value: 

[ORCON] Control group; odds ratios: 
Confidence Intervals: 
p-value: 


[ORADJ] Are the odds ratios adjusted? 
© 1. Yes. 
© 0.No 


Adjusted by (explain): 


[CHISC] X?2 value with af: 


[PAGEEFFECT] Number of the page from where you extract statistical data: 


e Effect sizes at follow-up 


[ESFOLLOW1] Calculated effect at follows up 1:_ 
[ESFOLL1] Number of months after intervention for follow-up 1: 


[ESFOLLOW2] Calculated effect at follows up 2: 
[ESFOLL2] Number of months after intervention for follow-up 2: 


[ESFOLLOW3] Calculated effect at follows up 3: 
[ESFOLL3] Number of months after intervention for follow-up 3: 


[ESFOLLOW4] Calculated effect at follows up 4: 
[ESFOLL4] Number of months after intervention for follow-up 4: 
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12.3 EPOC ‘RISK OF BIAS’ TOOL 


Study: 


Date: 


Coder: 


Item 


1 


200 


Criteria Evaluation 


Was the allocation sequence adequately generated? 

Score “Low risk” if a random component in the sequence generation 
process is described (eg., Referring to a random number table). Score 
“High risk” when a non-random method is used (eg performed by date 
of admission). NRCTs and CBA studies should be scored “High risk”. 
Score “Unclear risk” if not specified in the paper. 


Was the allocation adequately concealed? 

Score “Low risk” if the unit of allocation was by institution, team or 
professional and allocation was performed on all units at the start of the 
study; or if the unit of allocation was by patient or episode of care and 
there was some form of centralised randomisation scheme, an on-site 
computer system or sealed opaque envelopes were used. CBA studies 
should be scored “High risk”. Score “Unclear risk” if not specified in the 
paper. 


Were baseline outcome measurements similar?12 

Score “Low risk” if performance or patient outcomes were measured 
prior to the intervention, and no important differences were present 
across study groups. In RCTs, score “Low risk” if imbalanced but 
appropriate adjusted analysis was performed (e.g. Analysis of 
covariance). Score “High risk” if important differences were present and 
not adjusted for in analysis. If RCTs have no baseline measure of 
outcome, score “Unclear risk” 


Were baseline characteristics similar? 

Score “Low risk” if baseline characteristics of the study and control 
providers are reported and similar. Score “Unclear risk” if it is not clear 
in the paper (e.g. characteristics are mentioned in text but no data were 
presented). Score “High risk” if there is no report of characteristics in 
text or tables or if there are differences between control and 
intervention providers. Note that in some cases imbalance in patient 
characteristics may be due to recruitment bias whereby the provider 
was responsible for recruiting patients into the trial. 


Were incomplete outcome data adequately addressed?" 

Score “Low risk” if missing outcome measures were unlikely to bias the 
results (e.g. the proportion of missing data was similar in the 
intervention and control groups or the proportion of missing data was 
less than the effect size i.e. unlikely to overturn the study result). Score 
“High risk” if missing outcome data was likely to bias the results. Score 
“Unclear risk” if not specified in the paper (Do not assume 100% follow 
up unless stated explicitly). 
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Justification 


6 Was knowledge of the allocated interventions adequately 
prevented during the study?’ 
Score “Low risk” if the authors state explicitly that the primary outcome 
variables were assessed blindly, or the outcomes are objective, e.g. 
length of hospital stay. Primary outcomes are those variables that 
correspond to the primary hypothesis or question as defined by the 
authors. Score “High risk” if the outcomes were not assessed blindly. 
Score “Unclear risk” if not specified in the paper. 


7 Was the study adequately protected against contamination? 
Score “Low risk” if allocation was by community, institution or practice 
and it is unlikely that the control group received the intervention. Score 
“High risk” if it is likely that the control group received the intervention 
(e.g. if patients rather than professionals were randomised). Score 
“Unclear risk” if professionals were allocated within a clinic or practice 
and it is possible that communication between intervention and control 
professionals could have occurred (e.g. physicians within practices 
were allocated to intervention or control) 


8 Was the study free from selective outcome reporting? 

Score “Low risk” if there is no evidence that outcomes were 
selectively reported (e.g. all relevant outcomes in the methods section 
are reported in the results section). Score “High risk” if some important 
outcomes are subsequently omitted from the results. Score “Unclear 
risk” if not specified in the paper. 


12.4 TRICHOTOMOUS SCALE FOR POTENTIAL FINANCIAL 
CONFLICT OF INTEREST 


Study: Date: Coder: 
Financial COI Coding Rule Justification 
Unlikely None of the study authors is programme developer, 


collaborator of programme developer or license holder. 


Possible (Programme developer or collaborator of programme 
developer is study author) AND 
((programme is not (yet) commercially available) OR 
(business model is ‘not-for-profit’)) 


Likely (Programme developer or collaborator of programme 
developer is study author) AND (pra 
commercially available) 

AND (business model is ‘for profit’) 


*This instrument has been developed by Eisner & Humphreys (2012) 
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13. Electronic searches 


Australian Education Index 
Date: 05/10/ 2015 

Output of the searches: 148 
Saved hits: 1 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


British Education Index 
Date: 05/10/ 2015 

Output of the searches: 202 
Saved hits: 5 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school AND stand down 


BMJ Controlled Trials 
Date: 06/10/ 2015 
Output of the searches: 550 
Saved hits: 0 
Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 
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CBCA Education (Canada) 
Date: 14/10/2015 

Output of the searches: 5652 
Saved hits: 58 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school expelled 
4) school expulsion 
5) school stand down 


ClinicalTrials.gov 

Date: 07/10/ 2015 

Output of the searches: 285 
Saved hits: 3 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


Criminal Justice Abstract 
Date: 06/10/ 2015 

Output of the searches: 369 
Saved hits: 19 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


Cochrane Central Register of Controlled Trials (CENTRAL) 
Date: 16/10/2015 
Output of the searches: 154 
Saved hits: 12 
Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 
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Educational Resources Information Center - ERIC 
Date: 14/10/2015 

Output of the searches: 1491 

Saved hits: 48 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


Ethos Beta 

Date: 08/10/ 2015 

Output of the searches: 482 
Saved hits: 5 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


EMBASE 

Date: From 08/10/ 2015 to 12/10/2015 
Output of the searches: 1569 

Saved hits: 12 


Final searches string: 
1) "school exclusion" AND 
2) "school exclusion" AND evaluation 
3) "school exclusion" AND random* 
4) "school exclusion" AND intervention, 
5) "school exclusion" AND effective’, 
6) "school exclusion" AND efficacy, 
7) "school exclusion" AND quasi, 
8) "school exclusion" AND impact, 
9) "school exclusion" AND RCT, 
10) "school exclusion" AND school management 
11) "school exclusion" AND classroom management 
12) "school exclusion" AND school support project* 
13) "school exclusion" AND skills training 
14) "school exclusion" AND disciplinary methods 
15) "school exclusion" AND token economy 
16) "school exclusion" AND program* 
17) "school exclusion" AND *intervention* 
18) "school exclusion" AND strateg* 
19) "school exclusion" AND schoolchildren 
20)"school exclusion" AND *children* 
21) "school exclusion" AND school-age* 
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22)"school exclusion" AND adolescent* 

23) "school exclusion" AND pupil* 

24)"school exclusion" AND student* 

25) "school suspension" AND experiment* 

26) "school suspension" AND evaluation 

27) "school suspension" AND random* 

28) "school suspension" AND intervention, 

29) "school suspension" AND effective”*, 
30)"school suspension" AND efficacy, 

31) "school suspension" AND quasi, 

32)"school suspension" AND impact, 

33) "school suspension" AND RCT, 

34) "school suspension" AND school management 
35) "school suspension" AND classroom management 
36) "school suspension" AND school support project* 
37) "school suspension" AND skills training 
38)"school suspension" AND disciplinary methods 
39) "school suspension" AND token economy 

40) "school suspension" AND program* 

41) "school suspension" AND *intervention* 

42) "school suspension" AND strateg* 

43) "school suspension" AND schoolchildren 
44)"school suspension" AND *children* 

45) "school suspension" AND school-age* 
46)"school suspension" AND adolescent* 

47) "school suspension" AND pupil* 

48)"school suspension" AND student* 

49) "school expulsion" AND experiment* 
50)"school expulsion" AND evaluation 

51) "school expulsion" AND random* 

52) "school expulsion" AND intervention, 

53) "school expulsion" AND effective”, 

54) "school expulsion" AND efficacy, 

55) "school expulsion" AND quasi, 

56) "school expulsion" AND impact, 

57) "school expulsion" AND RCT, 

58) "school expulsion" AND school management 
59) "school expulsion" AND classroom management 
60)"school expulsion" AND school support project* 
61) "school expulsion" AND skills training 

62) "school expulsion" AND disciplinary methods 
63) "school expulsion" AND token economy 

64) "school expulsion" AND program* 

65) "school expulsion" AND *intervention* 

66) "school expulsion" AND strateg* 

67) "school expulsion" AND schoolchildren 

68) "school expulsion" AND *children* 

69) "school expulsion" AND school-age* 

70) "school expulsion" AND adolescent* 

71) "school expulsion" AND pupil* 

72) "school expulsion" AND student* 
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Google Scholar 

Date: From 27/10/ 2015 to 04/11/2015 
Output of the searches: 13525 

Saved hits: 165 

Final searches string: 


1) 
2) 


3) 


4) 


5) 
6) 


7) 
8) 


"school exclusion" AND experiment OR evaluation experiment OR evaluation OR 
quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random* 
"school exclusion" AND school management OR classroom management OR school 
support project* OR skills training OR disciplinary methods OR token economy OR 
program* OR *intervention* OR strateg* 

"school suspen*" AND experiment* OR evaluation OR quasi OR effective* OR RCT 
OR impact OR efficacy OR intervention OR random* 

"school suspen*" AND school management OR classroom management OR school 
support project* OR skills training OR disciplinary methods OR token economy OR 
program* OR *intervention* OR strateg* 

"school expulsion" AND experiment OR evaluation experiment OR evaluation OR 
quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random* 
"school expulsion" AND school management OR classroom management OR school 
support project* OR skills training OR disciplinary methods OR token economy OR 
program* OR *intervention* OR strateg* 

“stand-down” AND experiment* OR evaluation OR quasi OR effective* OR RCT OR 
impact OR efficacy OR intervention OR random* 

“stand-down” AND school management OR classroom management OR school 
support project* OR skills training OR disciplinary methods OR token economy OR 
program* OR *intervention* OR strateg* 


[In order to manage the searches we break down the searches by year. For instance 1980- 
1990, 1991-2005 and 2006 to date] 


Google 

Date: From 27/10/ 2015 to 04/11/2015 
Output of the searches: 4092 

Saved hits: 22 

Final searches string: 
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1) 


9) 


"school exclusion" AND experiment* OR evaluation OR quasi OR RCT OR impact OR 
efficacy OR intervention 

"school exclusion" AND random* 

"school exclusion" AND effective* 

"school exclusion" AND school management OR classroom management OR school 
support project* OR skills training OR disciplinary methods OR token economy 
"school exclusion" AND *intervention* 

"school exclusion" AND strateg* 

"school exclusion" AND program* 

"school suspension" AND experiment* OR evaluation OR quasi OR RCT OR impact 
OR efficacy OR intervention 

"school suspension" AND random* 


10) "school suspension" AND effective* 
11) "school suspension" AND school management OR classroom management OR school 


support project* OR skills training OR disciplinary methods OR token economy 


12) "school suspension" AND *intervention* 

13) "school suspension" AND strateg* 

14) "school suspension" AND program* 

15) school expulsion AND...intervention* OR strategy* OR program* 
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16) school expulsion AND...random* OR effective* 
17) school expulsion AND...school management OR classroom management OR school 


support project* OR skills training OR disciplinary methods OR token economy 


18) school expulsion AND...random* OR effective 
19) school expulsion AND...experiment* OR evaluation OR quasi OR RCT OR impact OR 


efficacy OR intervention 


Institute of Education Sciences — What Works Clearinghouse 
Date: 30/09/ 2015 

Output of the searches: 2 

Saved hits: o 

Final searches string: 


school exclusion 
school suspension 
school suspended 
school expelled 
school expulsion 
school stand down 


ISI Web of Knowledge 

Date: from 07/09/ 2015 to 17/09/2015 
Output of the searches: 4391 

Saved hits: 270 

Final searches string: 
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1) 


2) 


3) 


4) 


TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (in 
tervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPI 
C: Gmpact) OR TOPIC: (RCT) 

AND 

TOPIC: (school) AND TOPIC: (*exclusion) 

TOPIC: (school) AND TOPIC: (*suspension*) 

AND 

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (in 
tervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPI 
C: Gmpact) OR TOPIC: (RCT) 

TOPIC: (school) AND TOPIC: (suspended) 

AND 

TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (in 
tervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPI 
C: Gmpact) OR TOPIC: (RCT) 

Refined by: RESEARCH AREAS: (PSYCHOLOGY OR FAMILY STUDIES OR 
EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL SCIENCES OR 
PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR ETHNIC 
STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES OTHER 
TOPICS OR SOCIAL ISSUES) 

Timespan=1980-2015 

TOPIC: (school) AND TOPIC: (expelled) 

AND 
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TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (in 
tervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPI 
C: Gmpact) OR TOPIC: (RCT) 

5) TOPIC: (school) AND TOPIC: (expulsion) 
AND 
TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (in 
tervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPI 
C: Gmpact) OR TOPIC: (RCT) 

6) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC: (disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 


TOPIC: (school) AND TOPIC: (*exclusion) 
7) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC:(disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 


TOPIC: (school) AND TOPIC: (*exclusion) 
8) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC:(disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 


TOPIC: (school) AND TOPIC: (*suspension*) 
9) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC:(disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 


TOPIC: (school) AND TOPIC: (suspended) 
10) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC: (disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 


TOPIC: (school) AND TOPIC: (expelled) 
11) TOPIC: (school management) OR TOPIC: (classroom 
management) OR TOPIC: (school support project*) OR TOPIC: (skills 
training) OR TOPIC: (disciplinary methods) OR TOPIC: (token 
economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 
AND 
TOPIC: (school) AND TOPIC: (expulsion) 
12) TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school- 
age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student) 
AND 


TOPIC: (school) AND TOPIC: (*exclusion) 
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13) TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school- 
age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student) 


AND 


TOPIC: (school) AND TOPIC: (*suspension*) 

14) TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school- 
age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student) 
AND 


TOPIC: (school) AND TOPIC: (suspended) 

15) TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school- 
age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student) 
AND 


TOPIC: (school) AND TOPIC: (expelled) 
16) TOPIC: (school management) OR TOPIC: (classroom 

management) OR TOPIC: (school support project*) OR TOPIC: (skills 

training) OR TOPIC:(disciplinary methods) OR TOPIC: (token 

economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*) 

AND 

TOPIC: (school) AND TOPIC: (expulsion) 
All searches in ISI Web of Knowledge were Refined by: RESEARCH AREAS: (PSYCHOLOGY 
OR FAMILY STUDIES OR EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL 
SCIENCES OR PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR 
ETHNIC STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES 
OTHER TOPICS OR SOCIAL ISSUES) 
Timespan=1980-2015 


MEDLINE 

Date: 05/10/ 2015 

Output of the searches: 142 
Saved hits: o 


Final searches string: 
1) "school exclusion" AND experiment* 
2) "school exclusion" AND evaluation 
3) "school exclusion" AND random* 
4) "school exclusion" AND intervention, 
5) "school exclusion" AND effective™, 
6) "school exclusion" AND efficacy, 
7) "school exclusion" AND quasi, 
8) "school exclusion" AND impact, 
9) "school exclusion" AND RCT, 
10) "school exclusion" AND school management 
11) "school exclusion" AND classroom management 
12) "school exclusion" AND school support project* 
13) "school exclusion" AND skills training 
14) "school exclusion" AND disciplinary methods 
15) "school exclusion" AND token economy 
16) "school exclusion" AND program* 
17) "school exclusion" AND *intervention* 
18) "school exclusion" AND strateg* 
19) "school exclusion" AND schoolchildren 
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20)"school exclusion" AND *children* 

21) "school exclusion" AND school-age* 
22)"school exclusion" AND adolescent* 

23) "school exclusion" AND pupil* 

24)"school exclusion" AND student* 

25) "school suspension" AND experiment* 

26) "school suspension" AND evaluation 

27) "school suspension" AND random* 
28)"school suspension" AND intervention, 
29)"school suspension" AND effective”, 
30)"school suspension" AND efficacy, 

31) "school suspension" AND quasi, 

32)"school suspension" AND impact, 

33) "school suspension" AND RCT, 

34) "school suspension" AND school management 
35) "school suspension" AND classroom management 
36) "school suspension" AND school support project* 
37) "school suspension" AND skills training 
38)"school suspension" AND disciplinary methods 
39) "school suspension" AND token economy 

40) "school suspension" AND program* 

41) "school suspension" AND *intervention* 

42) "school suspension" AND strateg* 

43) "school suspension" AND schoolchildren 
44)"school suspension" AND *children* 

45) "school suspension" AND school-age* 
46)"school suspension" AND adolescent* 

47) "school suspension" AND pupil* 

48)"school suspension" AND student* 

49) "school expulsion" AND experiment* 
50)"school expulsion" AND evaluation 

51) "school expulsion" AND random* 

52) "school expulsion" AND intervention, 

53) "school expulsion" AND effective”, 

54) "school expulsion" AND efficacy, 

55) "school expulsion" AND quasi, 

56) "school expulsion" AND impact, 

57) "school expulsion" AND RCT, 

58) "school expulsion" AND school management 
59) "school expulsion" AND classroom management 
60)"school expulsion" AND school support project* 
61) "school expulsion" AND skills training 

62) "school expulsion" AND disciplinary methods 
63) "school expulsion" AND token economy 

64) "school expulsion" AND program* 

65) "school expulsion" AND *intervention* 

66) "school expulsion" AND strateg* 

67) "school expulsion" AND schoolchildren 

68) "school expulsion" AND *children* 

69) "school expulsion" AND school-age* 

70) "school expulsion" AND adolescent* 

71) "school expulsion" AND pupil* 

72) "school expulsion" AND student* 
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The National dropout prevention centre network 
Date: 16/10/2015 

Output of the searches: 26 

Saved hits: 0 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


The Netherlands Trial Register - NTR 
Date: 07/10/ 2015 

Output of the searches: o 

Saved hits: o 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


Open Grey 
Date: 07/10/ 2015 
Output of the searches: 169 
Saved hits: 3 
Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


ProQuest Dissertations & Theses A&I: Social Sciences 
Including: Dissertation Abstracts & Index to Theses Database 
Date: 19/10/2015 

Output of the searches: 5280 

Saved hits: 344 


Final searches string: 
1) "school exclusion" AND experiment 
2) "school exclusion" AND evaluation 
3) "school exclusion" AND random* 
4) "school exclusion" AND intervention 
5) "school exclusion" AND effective* 
6) "school exclusion" AND efficacy 
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7) "school exclusion" AND quasi 

8) "school exclusion" AND impact 

9) "school exclusion" AND RCT 

10) "school exclusion" AND school management 

11) "school exclusion" AND classroom management 
12) "school exclusion" AND school support project* 
13) "school exclusion" AND skills training 

14) "school exclusion" AND disciplinary methods 
15) "school exclusion" AND token economy 

16) "school exclusion" AND program* 

17) "school exclusion" AND strateg* 

18) “school suspension” AND experiment 

19) “school suspension” AND evaluation 

20) “school suspension” AND random* 

21) “school suspension” AND intervention 

22) “school suspension” AND effective* 

23) “school suspension” AND efficacy 

24) “school suspension” AND quasi 

25) “school suspension” AND impact 

26) “school suspension” AND RCT 

27) “school suspension” AND school management 
28) “school suspension” AND classroom management 
29) “school suspension” AND school support project* 
30) “school suspension” AND skills training 

31) “school suspension” AND disciplinary methods 
32) “school suspension” AND token economy 

33) “school suspension” AND program* 

34) “school suspension” AND strateg* 

35) “school stand-down” AND experiment 

36) “school stand-down” AND evaluation 

37) “school stand-down” AND random* 

38) “school stand-down” AND intervention 

39) “school stand-down” AND effective* 

40) “school stand-down” AND efficacy 

41) “school stand-down” AND quasi 

42) “school stand-down” AND impact 

43) “school stand-down” AND RCT 

44) “school stand-down” AND school management 
45) “school stand-down” AND classroom management 
46) “school stand-down” AND school support project* 
47) “school stand-down” AND skills training 

48) “school stand-down” AND disciplinary methods 
49) “school stand-down” AND token economy 

50) “school stand-down” AND program* 

51) “school stand-down” AND strateg* 

52) “school expelled” AND experiment 

53) “school expelled” AND evaluation 

54) “school expelled” AND random* 

55) “school expelled” AND intervention 

56) “school expelled” AND effective* 

57) “school expelled” AND efficacy 

58) “school expelled” AND quasi 

59) “school expelled” AND impact 

60) “school expelled” AND RCT 

61) “school expelled” AND school management 

62) “school expelled” AND classroom management 
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63) “school expelled” AND school support project* 
64) “school expelled” AND skills training 

65) “school expelled” AND disciplinary methods 
66) “school expelled” AND token economy 

67) “school expelled” AND program* 

68) “school expelled” AND strateg* 

69) “school expulsion” AND experiment 

70) “school expulsion” AND evaluation 

71) “school expulsion” AND random* 

72) “school expulsion” AND intervention 

73) “school expulsion” AND effective* 

74) “school expulsion” AND efficacy 

75) “school expulsion” AND quasi 

76) “school expulsion” AND impact 

77) “school expulsion” AND RCT 

78) “school expulsion” AND school management 
79) “school expulsion” AND classroom management 
80)“school expulsion” AND school support project* 
81) “school expulsion” AND skills training 

82) “school expulsion” AND disciplinary methods 
83) “school expulsion” AND token economy 

84) “school expulsion” AND program* 

85) “school expulsion” AND strateg* 

86) “school suspended” AND experiment 

87) “school suspended” AND evaluation 

88) “school suspended” AND random* 

89) “school suspended” AND intervention 

90) “school suspended” AND effective* 

91) “school suspended” AND efficacy 

92) “school suspended” AND quasi 

93) “school suspended” AND impact 

94) “school suspended” AND RCT 

95) “school suspended” AND school management 
96) “school suspended” AND classroom management 
97) “school suspended” AND school support project* 
98) “school suspended” AND skills training 

99) “school suspended” AND disciplinary methods 
100) “school suspended” AND token economy 

101) “school suspended” AND program* 

102) “school suspended” AND strateg* 


PsychINFO 

Date: From 12/10/ 2015 to 14/10/2015 
Output of the searches: 1538 

Saved hits: 86 

Final searches string: 
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1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 
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Sociological Abstract (ProQuest) 
Date: From 21/09/ 2015 to 23/09/2015 
Output of the searches: 2440 

Saved hits: 355 


Final searches string: 
1) "school exclusion" AND experiment* 
2) "school exclusion" AND evaluation 
3) "school exclusion" AND random* 
4) "school exclusion" AND intervention, 
5) "school exclusion" AND effective’, 
6) "school exclusion" AND efficacy, 
7) "school exclusion" AND quasi, 
8) "school exclusion" AND impact, 
9) "school exclusion" AND RCT, 
10) "school exclusion" AND school management 
11) "school exclusion" AND classroom management 
12) "school exclusion" AND school support project* 
13) "school exclusion" AND skills training 
14) "school exclusion" AND disciplinary methods 
15) "school exclusion" AND token economy 
16) "school exclusion" AND program* 
17) "school exclusion" AND *intervention* 
18) "school exclusion" AND strateg* 
19) "school exclusion" AND schoolchildren 
20)"school exclusion" AND *children* 
21) "school exclusion" AND school-age* 
22)"school exclusion" AND adolescent* 
23) "school exclusion" AND pupil* 
24)"school exclusion" AND student* 
25) "school suspension" AND experiment* 
26) "school suspension" AND evaluation 
27) "school suspension" AND random* 
28)"school suspension" AND intervention, 
29)"school suspension" AND effective’, 
30)"school suspension" AND efficacy, 
31) "school suspension" AND quasi, 
32)"school suspension" AND impact, 
33) "school suspension" AND RCT, 
34)"school suspension" AND school management 
35) "school suspension" AND classroom management 
36)"school suspension" AND school support project* 
37) "school suspension" AND skills training 
38)"school suspension" AND disciplinary methods 
39)"school suspension" AND token economy 
40)"school suspension" AND program* 
41) "school suspension" AND *intervention* 
42)"school suspension" AND strateg* 
43)"school suspension" AND schoolchildren 
44)"school suspension" AND *children* 
45) "school suspension" AND school-age* 
46)"school suspension" AND adolescent* 
47) "school suspension" AND pupil* 
48)"school suspension" AND student* 
49) "school expulsion" AND experiment* 
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50)"school expulsion" AND evaluation 

51) "school expulsion" AND random* 

52) "school expulsion" AND intervention, 

53) "school expulsion" AND effective’, 

54) "school expulsion" AND efficacy, 

55) "school expulsion" AND quasi, 

56) "school expulsion" AND impact, 

57) "school expulsion" AND RCT, 

58) "school expulsion" AND school management 
59) "school expulsion" AND classroom management 
60)"school expulsion" AND school support project* 
61) "school expulsion" AND skills training 

62) "school expulsion" AND disciplinary methods 
63) "school expulsion" AND token economy 

64) "school expulsion" AND program* 

65) "school expulsion" AND *intervention* 

66) "school expulsion" AND strateg* 

67) "school expulsion" AND schoolchildren 

68) "school expulsion" AND *children* 

69) "school expulsion" AND school-age* 

70) "school expulsion" AND adolescent* 

71) "school expulsion" AND pupil* 

72) "school expulsion" AND student* 


SciElo 

Date: 06/10/ 2015 
Output of the searches: 32 
Saved hits: 0 


Final searches string: 
1) "school exclusion evaluation" 
2) "school exclusion experiment" 
3) "school suspension evaluation" 
4) "school suspension experiment" 
5) "school expulsion evaluation” 
6) "school expulsion experiment” 
7) suspension escolar 
8) suspension escolar evaluacion 
9) suspension escolar experiment 
10) expulsion escolar 
11) expulsion escolar evaluacion 
12) expulsion escolar experiment 
13) expulsion escuela 


Sciencegov 

Date: From 05/10/ 2015 to 06/10/2015 
Output of the searches: 319 

Saved hits: 61 


Final searches string: 
1) "school suspension" AND evaluation 
2) "school suspension" AND experiment* 
3) "school exclusion" AND evaluation 
4) "school exclusion" AND experiment* 
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5) "school expulsion" AND evaluation 
6) "school expulsion" AND experiment* 


The Campbell Collaboration Social, Psychological, Educational and 
Criminological Trials Register (C2-SPECTR) 
Date: 16/10/2015 
Output of the searches: 1 
Saved hits: 1 
Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 


5) school expulsion 
6) school stand down 


Trials Journal 

Date: 06/10/ 2015 
Output of the searches: 56 
Saved hits: 1 


Final searches string: 
1) school exclusion 
2) school suspension 
3) school suspended 
4) school expelled 
5) school expulsion 
6) school stand down 


WHO.- International Clinical Trials Registry Platform (ICTRP) 
Date: 30/09/2015 

Output of the searches: 3 

Saved hits: 3 


Final searches string: 
1) "school exclusion" (other outcome terms searched with no results) 
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CampbellCollaboration 


About this review 


School exclusion is associated with undesirable effects on developmental outcomes. It 
increases the likelihood of poor academic performance, antisocial behavior, and poor 
employment prospects. This school sanction disproportionally affects males, ethnic minorities, 
those who come from disadvantaged economic backgrounds, and those with special 
educational needs. 


This review assesses the effectiveness of programmes to reduce the prevalence of exclusion. 
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